Reasoning Is a Scaffold
Chain-of-thought buys the model room to work, and self-consistency votes across sampled paths. Only a check outside the model can fail a wrong answer.
Chain-of-thought buys the model room to work. Only a check outside the model turns that room into a correct answer.
Before an agent touches a tool, it has to work through a hard task without drifting. That is the oldest problem in AI, and tool wiring does not skip it.
The map note draws the loop in five steps: observe, decide, act, examine, remember. Reasoning improves the decide step, and it leaves the other four alone.
Intermediate Steps Change the Continuation
A language model is trained to produce a plausible continuation. Truth is not the training target. Intermediate steps change what a plausible continuation has to contain. The model builds the answer across many tokens rather than one.
The steps are a scaffold. They give the model somewhere to allocate attention and hold partial results.
Last-letter concatenation shows the flip. Ask for the answer and the model treats the task as a pattern over names. Show the decomposition and the same model runs a sequence of tiny operations. The task goes from brittle to reliable.
Self-Consistency Is a Vote Over Paths
Sample many reasoning paths instead of one, then pull the final answer out of each. Let the answers vote and take the winner.
A single path fails for local reasons. One early token that misleads drags the rest of the solution with it. Many samples give you several attempts at the same latent process, and agreement across them raises confidence.
Ten kids run a maze and seven come out the same door. That door is probably the exit. The vote carries no guarantee, and it still beats the first kid who ran.
Analogical Prompting Asks the Model to Build Its Own Examples
Analogical prompting hands the example writing to the model. Ask it to recall or invent related problems, solve those first, then carry the shape to the real one.
Hand-written examples cost you time, and each one covers a single shape. A tutor picks one close cousin, names the shape, and lets the student carry it to the rest.
Correction Needs a Signal From Outside the Model
Self-correction is no safety net. Ask a model to review its own answer and it can repair a wrong one. It can also break a right one. An instruction to check your work does not create ground truth.
A trace creates intermediate state, and that state is not automatically correct. A fluent chain can drift or carry a wrong early assumption all the way to the answer. The signals that catch it are cheap and specific.
- Unit tests for code
- A calculator for arithmetic
- A symbolic solver for constraints and a verifier for proofs
- Retrieval for facts
- Environment feedback for plans
- A human for judgment
When a reasoning step can change an action, you must be able to examine that step.
Later notes build these signals into the loop. The software note uses tests, and the neural-symbolic note uses a solver. The scaling-policy note carries the same distinction to the release gate.
Agents Multiply Reasoning Mistakes
A wrong answer is one bad output. A wrong plan calls tools, writes files, spends money, emails people, and pushes a bad decision downstream.
The fix is a bound on what a wrong chain can touch. Four moves put that bound in place.
- Trace the run so each step is visible
- Sample so that no single path gets your trust
- Run the verifier before the action
- Stop the run when the verifier fails
An agent that keeps going past a failed check does not have a check.
The Builder Test
Take one task your agent gets wrong today. Add the cheapest check that lives outside the model, a test or a calculator. Run the same set of tasks both ways and count what the check caught.
Compare five shapes on the same tasks: short, decomposed, tool-verified, search-guided, and one with the scratch work hidden. Score every shape on the rate of errors that matter.
Read each intermediate step on its own. A step you cannot examine alone is a step you are trusting. If you cannot say what changed after each step, the run is not observable enough to test.
Check that your decomposition matches the real structure of the task. If a verifier catches more of those errors than a longer answer, ship the verifier.
What Carries
A reasoning trace earns its cost when it improves a decision or exposes something you can examine. Length proves nothing on its own. A long explanation can hide anchoring, circular justification, and an early mistake the chain preserves.
A verifier lives outside the model, so the agent needs a way to reach it. That is the act step.