Grounding with Retrieved Context
Every technique so far has assumed the model already knows what it needs to answer well, it just needed the right tokens in front of it to use that knowledge properly. Retrieval-augmented generation, RAG, breaks that assumption on purpose. Sometimes the model genuinely doesn't know the answer, because the fact you're asking about is private, obscure, or newer than its training data, and no amount of clever instruction wording fixes that. What does fix it is retrieving the relevant text from somewhere else and splicing it directly into the context window, which is really just lesson 1's mechanism again, more tokens for attention to draw on, except this time those tokens are facts instead of instructions or examples.
Predict before you look
The Solstice desk lamp is a fictional product invented for this lesson, so the model has never seen a real spec sheet for it. Before you look, when asked how long its battery lasts on the lowest setting with no supporting information given, do you expect the model to confidently state a number, or to say it doesn't know?
Without anything to go on, the model didn't invent a number, it said plainly that it wasn't aware of the Solstice desk lamp at all and asked for more context. That's actually the responsible answer, and it's worth sitting with for a second, because a less careful model could have just as easily guessed a plausible-sounding figure in the exact same confident tone. Either way, though, you're left without what you actually needed, an answer.
Give the exact same model a short, three-sentence spec sheet excerpt spliced into the prompt, and the question that was unanswerable a moment ago gets a precise, correct response, 14 hours, read straight out of the retrieved text. Nothing about the model changed between the two runs. What changed is what showed up in its context window before it had to answer. That's the whole idea of RAG, retrieve the passage that actually contains the fact, put it in front of the model, and let attention do what it already does with any other token.
One honest limitation of this lesson, on purpose, is that we handed the model the right passage ourselves. Deciding what to retrieve, when a first retrieval attempt wasn't good enough, and whether to search again before answering, that's a real decision an agent has to make on its own, and it belongs in the Agentic AI course rather than here. This lesson is about the mechanical half of retrieval, the part that's still just prompt engineering.
What's the most accurate description of what changed between the two runs above?