From Completion to Chat: Instruction Tuning

Every lesson so far has trained a model on one objective: predict the next token, given raw text. Do that at massive scale on a huge slice of the internet, and you get something genuinely powerful — but not something that reliably answers questions or follows instructions. It just continues text in whatever pattern it's already in. That gap is real, and it's worth seeing directly before talking about how it gets closed.

A model trained purely to continue internet text has seen plenty of text that starts with something like "Explain photosynthesis" — homework assignments, worksheets, quiz banks — and statistically, that phrase is often followed by more instructions, not a direct answer. The model isn't being unhelpful on purpose; it genuinely doesn't know the prompt is supposed to be treated as a command rather than a pattern to extend.

Instruction tuning (often called supervised fine-tuning, or SFT) fixes this the same way every fine-tuning lesson in this course already works: keep training the model, but on a new, curated dataset — this time, thousands of hand-written or hand-selected (instruction, response) pairs, where the response is a genuinely helpful, direct answer to the instruction.

One real technical detail worth knowing: during this training, the loss is typically only computed on the response portion of each example, not the instruction. The model isn't being taught to predict instructions — it already knows how to produce text; it's being taught what a good response to an instruction looks like.

Instruction tuning teaches the model the right shape of behavior — respond, don't continue — using examples with one clearly correct answer per prompt. But plenty of real questions don't have one obviously correct response; they have several plausible ones that differ in tone, helpfulness, or safety. Capturing which one people actually prefer needs a different kind of training signal entirely — that's next.