LoRA with your own tiny language model

A “language model” here is a table of scores W: for each current word, how much it likes each next word. You choose the words, what the original model prefers, and what fine-tuning should teach it. We freeze W and train only a small adapter ΔW = B·A.

Set up

Dictionary: 3 to 8 words, separated by commas. Presets (they reset the words too):

Adapter rank r = 1 (max rank of ΔW) Learning rate = 1.0

Loss (avg KL to target): –

Trainable numbers: –

rank(ΔW): –

rank of the change needed: –

What the model predicts

Each group of bars is the next-word probability after the word on the left, in dictionary order. The orange bar marks the word you’re teaching it to say.

Original:

Adapted:

Target:

Each line follows the single most likely next word, starting from the first word in your dictionary.

What the adapter is made of

Each column of B times its row of A is one simple “layer” of change; the layers add up to ΔW. You can edit B and A by hand (dashed cells).

Each little grid has rows = current word and columns = next word, in dictionary order. Orange adds to a score, blue subtracts. “needed” is the change that would fully produce the target.

A rule of thumb to test

The rank you need is roughly the number of different words you teach it to say. “Always say arr” is one word, so rank 1 is enough. Teach it two different words and you need rank 2. Six different rewrites need rank 6. Try each preset with r too low, then raise r.