tenseleyflow/dlm-vsc / b1d46a6

Browse files

Replace preference section with instruction for SFT-only training

Authored by mfwolffe <wolffemf@dukes.jmu.edu>
SHA
b1d46a60e9162fd56ad68681907ea0bb10098984
Parents
3dc4225
Tree
b1f9f21

1 changed file

StatusFile+-
M test/fixtures/sample.dlm 5 8
test/fixtures/sample.dlmmodified
@@ -27,12 +27,9 @@ How do you train a DLM?
2727
 ### A
2828
 Run `dlm train your-file.dlm` and the adapter trains on the document content.
2929
30
-::preference::
31
-### Prompt
32
-Explain LoRA in one sentence.
33
-
34
-### Chosen
35
-LoRA adds small trainable matrices to frozen model layers, enabling efficient fine-tuning.
30
+::instruction::
31
+### Q
32
+What is LoRA?
3633
37
-### Rejected
38
-LoRA is a method for training language models that involves modifying the architecture of the model by introducing additional parameters in the form of low-rank decomposition matrices that are applied to the attention weight matrices, which allows for parameter-efficient fine-tuning while keeping the original pre-trained weights frozen.
34
+### A
35
+LoRA adds small trainable matrices to frozen model layers, enabling efficient fine-tuning without modifying the full model weights.