Self-Consistency & Structured Output

Zero to GPT's sampling lesson covered temperature: at temperature 0 a model is deterministic (same input, same output, every time); above 0, sampling introduces real randomness, and the same prompt can produce genuinely different reasoning paths on different runs. Self-consistency turns that randomness into a strength: instead of trusting one sample, generate several independent ones and take the majority answer.

All five samples reasoned through the problem slightly differently — different phrasing, different order of stating intermediate numbers — but converged on the same final answer, 12. That convergence is itself informative: when independent reasoning paths agree, you have real evidence the answer is right, not just a single confident-sounding guess. If two or three of the five had landed on a different number, majority vote would have caught the outlier — a wrong reasoning chain that, sampled once, would have looked exactly as confident as a right one. Self-consistency's whole value proposition is trading extra inference calls for exactly that safety net.

A different, more mundane reliability problem: getting a model's output into an exact shape your code can parse. This is the same lesson as zero-shot vs. few-shot from lesson 2, applied to a schema instead of a label.

Without a schema, the model invented its own reasonable-but-unpredictable JSON shape — a pros array that wasn't asked for, a max_rating field split out from rating, and the whole thing wrapped in a markdown code fence. With an exact schema spelled out in the prompt (field names, types, and "reply with only the JSON"), the output matched precisely — no code fence, no extra fields, ready to parse. If your code depends on a specific shape, specifying that shape in the prompt is far more reliable than hoping the model infers it.