Prompt injection is the native attack on any system that mixes instructions with untrusted text. Every agent in this course mixes them. The model reads a system prompt, a user request, a retrieved page, and a tool result in one window. Nothing in that window says which of them can give orders.

The release-gate note asked for a measurement that can change a launch decision. The text that arrives after the launch comes from a page the agent opened by itself. Someone wrote that page to reach your tools.

Trust Is Several Properties That Interact

Trust is a set of properties: privacy, behavior under attack, fairness, transparency, accountability, staying on the assigned goal, and resistance to misuse. One score hides which of them broke. For an agent they break through each other.

Retrieval carries the privacy failure. A hostile webpage breaks the behavior you tested, and tools turn a harmful plan into an executed one. Trust is the operating envelope for autonomy, and the envelope sets how far the agent goes before a human signs.

Agents Leak Through Their Tools

The leaks stack. A model memorizes pieces of its training data, and a system leaks private context through the window it assembles. An agent adds its own path: a secret handed to a tool, or one user's context carried into another user's turn.

State is the safety feature, and a shared context window is shared state. Data minimization does the heavy work, because an agent cannot leak a record it never reached.

Access control per user, redaction on the way out, and isolation between sessions hold the rest. Logs name which tool touched what data. Tests that deliberately try to extract sensitive information tell you whether the posture holds.

Prompt Injection Is the Native Agent Attack

Hostile text arrives on a webpage, inside a retrieved document, or in a tool result. A user probing the policy is the mild case. The hard case is text written for the agent and never shown to the human.

The loop note named five steps: observe, decide, act, examine, remember. Injection enters at observe, and the damage lands at act. Between those two steps, nothing asks who wrote the sentence the agent is following.

System instructions, developer policy, user intent, retrieved documents, webpages, emails, and tool outputs are different kinds of text. An agent that flattens them into one stream obeys whichever sentence arrived last.

• Quote and summarize untrusted content as evidence, with its source label attached.
• Tools declare their permissions and side effects before the agent calls them.
• A sensitive action stops for confirmation at the moment it runs.
• Logs keep a record of which instruction source caused which decision.

A suite built from normal prompts passes without touching any of this. The risk appears when the agent reads hostile text while holding tools, memory, private context, and authority to act.

The Same Capability Serves Both Sides

A model that finds a bug for you finds one for an attacker. Defenders get bug discovery, alert triage, and automated response. Attackers get phishing, reconnaissance, exploit generation, and social engineering in volume.

The net effect depends on capability, access, monitoring, policy, and the speed of defensive adoption. The last term is the one your team sets.

A Trustworthy Agent Hands You Evidence

The agent earns trust with a record: what it read, what it did, what it refused, what it doubts, and what needs a human signature. Tone proves nothing.

The workflow note called this a tape: what the agent relied on, what it changed, and how you undo it. Injection makes the tape load-bearing, because the tape holds the sentence that redirected the run. A more capable agent raises the value of an inspectable trail.

Safety measured as refusal behavior measures the wrong surface. The dangerous failures come from allowed actions: weak evidence, stale context, poisoned instructions, or a stop that came too early.

Policy Needs Evidence That Maps to Harms

Policy has to be science-based and evidence-based, and the phrase carries work. Written from vibes, it lags reality. Written from benchmark theater, it overfits the wrong signal, and it does that with a chart.

The evaluation that counts maps to an actual harm and an actual mitigation. An injection evaluation clears that bar when a failed run costs the agent a tool permission.

The Builder Test

Take one agent you run and list every text source it reads.

• Mark which sources can instruct the loop, and which stay evidence for the whole run.
• List the tools that reach private data, and the ones that run without a permission check.
• Name the actions that stop for a human, and the uncertainty that forces escalation.
• Write down how the system fails, and where that failure becomes visible to a person.

Then poison a page yourself, point the agent at it, and read which label failed.

What Carries

Every note in this course repaired one joint of the loop. Reasoning sharpened decide and tools built act. Grounding and the tape held examine and remember, and the release gate decided whether the loop ships at all.

The boundary between instructions and evidence guards observe. An outsider can write to that step, and a poisoned observation spends your tools. Agent safety starts when you decide what text can control the loop. Decide it before a webpage decides it for you.

Governance works at one moment: a measurement crosses a threshold and the release stops. Models keep scaling and the risks scale with them, so build that moment before you write another line of policy.

You Cannot Govern a Capability You Cannot Elicit

You cannot govern a capability you cannot elicit, measure, and track. A weak scaffold hides capability, and so does a lazy prompt. The model keeps what the test missed, and the first person outside your team with a better scaffold finds it.

Start by naming the dangerous capability in plain words. A vague worry has no eval and no threshold, so it never reaches a decision. Name the capability, then name how you elicit it.

Order decides whether the policy does any work. A policy written after the technical plan is fixed can only describe the plan. The measurement has to arrive early enough to change what ships.

Anthropic's Responsible Scaling Policy ties safety levels to capability thresholds and to the safeguards each level needs. The rule is a procedure: measure the capability, map it to a risk level, add the safeguards it needs. The threshold defines what changes behavior, and the mitigations must exist before deployment continues.

Elicitation Is a Stress Test

A weak evaluation misses a real capability because it asks badly. Strong elicitation gives the model its best reasonable shot: better prompts, more attempts, and expert-designed tasks. Tool access goes in when the task needs it.

A comfortable safety test measures your prompt. A strong one measures the model. You look for what the system can do before the world finds it for you.

The reasoning note kept verification outside the model. Elicitation is that verification, aimed at the ceiling of the model rather than at a single answer.

Measure the Model and the Deployed System

A model can be capable of a dangerous action and still be unlikely to take it inside a locked-down product. The same model can look harmless in a chat window and turn risky the moment you hand it tools.

So the policy carries two numbers. One is raw capability. The other is what the deployed system permits. That number moves with the scaffold, the permissions, the policy layer, the monitoring, and the human review path.

Tools move both numbers at once. Tool access raises what elicitation can show, and it raises what the product lets the model do. Hold both numbers, or you are guessing.

A Gate Has Four Pieces

A working gate has four pieces: a capability eval, a threshold, a mitigation, and an owner who can refuse. Take away the threshold and the measurement becomes trivia. A threshold with no required mitigation is theater. Without an owner, nobody pays a cost when someone says ship anyway.

I do not trust a gate until I can name the person who can refuse. The four pieces only work as a path, and the path ends at that name.

Agents Turn a Bad Answer Into a Bad Sequence

A chat model produces a bad answer. An agent produces a bad sequence. It gathers information, calls tools, makes irreversible changes, and adapts around weak guardrails.

So the safety review covers the whole sequence. You check four things in the trajectory: approval points, tool boundaries, stop conditions, and logs a human can follow. The stop condition carries the most weight, because uncertainty rises inside a run and the agent keeps acting.

Those logs carry the replayable trail from the enterprise note. The trail holds what the agent relied on, what it changed, and how a person undoes it. Without that trail, an incident review has nothing to replay.

Computer-use agents turn language into action on a screen, which is where the stakes rise. The capability and the risk come from the same mechanism. A gate for that system prices actions.

Governance Needs Layers Because Any Single Test Bends

Manual review alone cannot cover every release, so governance needs benchmarks. People can game a fully automated eval, and an automated eval can miss the risk that matters. Each test bends in a different direction, which is the argument for stacking them.

The working stack is automated benchmarks, red-team exercises, expert review, staged deployment, monitoring, and stop conditions a person can trigger. One decision procedure ties them to deployment limits, independent review, incident response, and public commitments.

Every layer produces a score, and the score expires. Treat a score as permanent and you are governing an old model. Capability moves with fine-tuning, tools, scaffolds, prompts, deployment context, and user incentives.

So write retest triggers into the policy. A new tool, a fine-tune, or a wider deployment voids the last score, and someone runs the eval again.

The Builder Test

Trace the stop path on a system you run now.

• Pick one eval whose result can stop your next release.
• A result crosses the threshold. Write down who hears about it first.
• Say which mitigation becomes mandatory before the release moves again.
• List what you examine a second time after the mitigation lands.
• Put a name on the person who can refuse, and say what refusing costs that person.

An empty line names the missing piece of your gate. Fill it before the model or the deployment changes again.

What Carries

Elicit hard, measure the model and the product it lives inside, then give one threshold authority over the release date. Everything else is documentation around a launch path someone already chose.

A gate decides what ships. The last note in this series meets the text that arrives after it ships.

Two teams can run the same model and report different scores. The scaffold around the model produced the gap. That makes the scaffold part of the result, and disclosure part of the science.

The scaffold is the loop from the agent-loop note. It runs observe, decide, act, examine, then remember. Every joint in it moves a score, and a leaderboard shows none of them.

Three Levels of Access, and One Supports Science

Open covers three kinds of release, and they differ by what an outsider can inspect.

• Black-box API. A user calls the model and cannot inspect the weights or the training process.
• Open weights. A user runs and adapts the weights, while the data and the training details stay closed.
• Fully open release. Weights, data, code, recipes, and reporting are public enough for outside scrutiny.

Open weights help a team run and adapt a model. The data shows what shaped it, and the recipe lets a stranger repeat the run. Published evaluations let an outsider prove the claim instead of trusting the release note.

A model can be useful to run and still be hard to audit or repeat. Treating open release as a slogan blurs five separate things. Reproducible science, useful artifacts, commercial reuse, safety review, and downstream accountability each need a different layer.

Agents Add a Second Black Box

An agent is a system. When the model is closed and the scaffold is closed, a benchmark compares two products. The number ranks the products and leaves the mechanism unnamed.

An open system lets a researcher ask a sharper question. Which part helped: the model, the tool interface, the retrieval source, the scoring code, or the search budget? Each answer sends the team to different work.

The reasoning note drew a line: reflection lives inside the model, and verification lives outside it. A closed scaffold pulls verification back inside the vendor, where no outsider repeats it.

For Agents, Openness Has More Layers Than Weights

Sort what a release can disclose into four groups.

• Model. Weights, tokenizer, architecture, and the training recipe where a team can publish it.
• Data. Pretraining data, instruction data, filtering choices, and known contamination checks.
• Scaffold. Prompt, tool definitions, memory policy, retrieval corpus, retries, action budget, stopping rule, and the code that runs the loop.
• Evaluation. Task set, environment version, scoring code, and failure analysis.

The scaffold row is where an agent score moves. Tool definitions are the action interface, and the memory policy is the state the loop carries. The stopping rule decides when a run ends.

One release can be open for use, closed for provenance, strong for adaptation, and weak for accountability at once. A score reported without those four groups is hard to read.

Two Teams, One Model, Different Scores

With a plain model, a team compares prompt in and answer out. With an agent, the scaffold joins the measured system.

The gap between two labs on the same weights comes from five knobs. Tool surface, attempts, context budget, retrieval, and environment each move the number. A cleaner browser interface and a hidden retrieval source both show up as model quality. None of the five touches the weights.

A team that states the knobs publishes evidence. A team that hides them publishes a product claim. Both ran the same model, and only one said what else it ran.

The demo-thinking note named the quiet version of this. A human supplied the missing state, and the system took the credit. An undisclosed scaffold does the same thing on a leaderboard.

Opacity Slows the Field Down

Two costs land right away. A team repeats a mistake because the recipe stayed private. A benchmark drifts into marketing because hidden details decided the result.

The third cost lands on safety. An agent acts through tools, so its trail matters more than the trail of a model that only answers. An outsider cannot audit what a release does not show.

The enterprise-agent note made every action name what it relied on, what it changed, and how to undo it. A release owes an outsider the same three answers.

The honest trade runs between scientific inspectability, competitive advantage, misuse risk, and privacy. Serious people draw that line in different places. A team that closes a layer and names it stays honest.

The Builder Test

Ask what a stranger can prove without help from the team that shipped the model.

• Run the model from the released artifact alone.
• Inspect the data: what went in, what got filtered out, which contamination checks ran.
• Repeat the recipe closely enough that the numbers land in the same place.
• Compare evaluations: task set, environment version, scoring code, and the failure cases.
• Read the license and the safety report, then name what each one forbids.

Whatever fails that list is the part of the release that only the team can vouch for. Add one line to the release note: who maintains this, and for how long.

What Carries

Openness is a stack of evidence. One label on a release tells a reader nothing about which layer opened.

Calling an API and reproducing a result are different claims. Both can be true, and only the second one lets the field build on the work.

Disclosure tells an outsider what happened. A release gate decides whether it happens at all.

A robot motion has no undo. That one constraint reorganizes everything the earlier notes in this unit built. Three parts carry the weight: a data pyramid, a simulator for cheap failure, and feedback that reports what changed.

The overview note drew the agent as five steps: observe, decide, act, examine, remember. Robotics keeps all five steps and makes the act step irreversible.

A Robot Motion Has No Undo

You can undo a browser click. A robot motion can break an object, hurt a person, or damage the robot itself. That changes the engineering.

A bad plan shows up as a missed grasp, a spilled object, an unsafe motion, or a broken recovery. A text agent is wrong on a screen. A robot is wrong in the room.

The embodied stack must turn high-level intent into low-level control under uncertainty. A language model can read "pick up the cup". The controller decides grip, force, trajectory, timing, contact, and recovery.

The Generalist Robot Needs More Than a Language Model

The target is one system that understands instructions, perceives the world, plans actions, and controls a body across many tasks.

That needs perception, control, simulation, imitation, reinforcement learning, and a way to move behavior from synthetic worlds onto real hardware. Foundation agents put broad priors and task-specific adaptation in one stack. It is the foundation-model bet with physics in the loop.

The agent-loop note put the rule as the weakest interface setting the ceiling. Here the weakest interface is a hand closing on a real object. A better language model does not close a gripper more gently.

Every Level of the Data Pyramid Pays for Something Else

The pyramid sorts training data by cost. At the base sits a large amount of cheap synthetic or weak data. Higher up the volume drops and the realism rises. At the top sit expensive real-world demonstrations.

The pyramid answers where the data comes from, and four sources fill it.

• Web video gives breadth.
• Simulation gives cheap variation.
• Human demonstrations give intent.
• Robot trajectories give contact, timing, and failure.

Real data is precious, and simulation is how you multiply it. Language and video data supply the broad priors under both. A robotics system uses every level to survive the gap between language and action.

Simulation Is a Rehearsal Room

A simulator lets the agent fail for free. Reinforcement learning shapes whole-body control, and imitation learning copies demonstrations. Domain randomization makes a policy less brittle when reality stops matching the simulator.

The failure mode is trust in a clean simulator. A policy can score well there and fall apart when lighting, friction, object shape, camera angle, or timing changes. Sim-to-real is the bridge between cheap practice and useful action.

Language Says What Matters, Control Does the Work

Language earns its place as the interface for goals, instructions, explanations, memory, and compositional tasks.

A low-level controller moves a gripper. A language-conditioned agent ties that motion to "clear the table, but leave the medicine bottle where it is". That is the jump from control policy to useful assistant.

The exception must live in state, not in the prompt. The agent-loop note called state the safety feature, and a medicine bottle is what that means with a body attached. If the constraint lives only in the sentence, the controller has nothing to check before it closes on the bottle.

Language tells the system what matters. The controller still owns everything below the sentence. The shallow read imagines instruction following as the hard part. The danger is a stack that treats a good instruction as a solved control problem.

The Builder Test

Name what the robot can examine after each action. A system that cannot tell whether the drawer opened, the object moved, or the grip failed cannot recover.

For a reversible action the examine step follows the act step. For an irreversible one it must come first. A safety check that runs after the motion is a log entry.

Separate perception, world state, planning, control, feedback, latency, and safety margin. On real hardware the agent survives noisy sensors, delayed feedback, irreversible actions, and an environment that changes after each move.

Four checks before the policy leaves the simulator:

• Name which state the robot senses and which state it infers.
• Name the actions that cannot be recovered from.
• Show how you tested the shift out of simulation.
• Point at the line where the safety constraint is enforced.

What Carries

Embodied intelligence is perception, action, feedback, and correction under real constraints. Action becomes part of the data distribution. A language plan is one component inside a closed loop.

Every claim in this unit rests on somebody being able to check it. The last three notes ask who checks yours.

Language handles messy intent. A solver handles exact constraints. The translation between them is the step that breaks. A solver can be perfectly wrong about a badly stated problem.

Language models are not good at every kind of planning. They sound coherent and violate constraints in the same paragraph.

One Missed Constraint Kills the Plan

Travel planning makes the failure obvious. One plan must satisfy dates, budgets, locations, preferences, availability, and ordering at the same time. Miss one and the itinerary is impossible to execute, however well it reads.

A plan that reads well and breaks one rule is worse than a plan that fails on its face. The first one gets executed.

Three Ways Out, and One Works Now

Researchers take three routes out of this.

• Make the language model better at planning on its own.
• Combine the language model with a symbolic solver.
• Train models so that symbolic structure emerges inside the network.

The middle route is the one a team can build this quarter. Route one waits for a better model, and route three waits for a training run. The middle route needs an interface between two components you already have.

A Solver Holds Every Constraint at Once

A symbolic solver is boring in the best way. Formalize the constraints correctly and the solver does not drop one because that sentence sat far away in the prompt. It searches the space with rules.

The split is clean. The model translates natural language into a structured problem. The solver searches that space and returns a valid plan. The model translates the result back into language a person can act on.

The reasoning note put reflection inside the model and verification outside it. The solver is that outside check, aimed at constraints instead of arithmetic.

Formalization Is the Fragile Step

The hard part is the translation from messy language into the correct formal problem. A user asks for something cheap but not terrible. The system must decide what cheap means and what terrible means. The system must also decide whether either one is a hard constraint or a soft preference.

A solver enforces whatever it is handed. Nothing inside it knows the difference between a constraint the user meant and one the model invented.

The tools note said it plainly: the weakest interface sets the ceiling. Here the weakest interface is the sentence that becomes a constraint list.

The model still earns its place at that boundary. It handles the ambiguity and asks the question that settles it. Then it hands over a structure the solver can search.

I review the constraint list, not the plan. Expose the structured object before the solver runs. Show the list in the user's own words, and mark each constraint hard or soft. Let a person or a test reject a malformed structure before any action runs.

Catch a wrong list and the fix costs one sentence. Miss it and the fix costs whatever the plan already committed.

Solver Traces Teach the Search Path

The third route becomes practical here. SearchFormer and DualFormer-style work trains on solver traces, and uses search dynamics and reasoning traces to teach better planning behavior.

The model sees more than the final answer. It sees the path that a stronger search process took.

A hybrid system writes those traces while it runs. The training data for route three is a by-product of route two.

Use the Pattern Where Correctness Is Checkable

Use the hybrid pattern when correctness depends on constraints you can examine. Schedules, routes, resource allocation, budgets, dependencies, access policies, inventory, and proof-like structure all qualify.

Skip it when the real problem is taste, strategy, judgment, or open-ended synthesis. Solvers can help at the edges there, and they will not define the target for you.

The pattern also gets thin when you ask the model to reason through everything in prose. That breaks when the environment has hard rules, long horizons, or costly invalid actions.

The Builder Test

Take one task you own this week and draw it. Label the part that reads ambiguity and the part that searches or plans. A single component doing both is the joint that fails first.

Name the part of that task that must be exact. Put it behind a tool, solver, verification step, or safety gate. Do not hope that the model stays careful.

• List the constraints that cannot be violated, before anything is generated.
• Name the one place where learned ranking beats a rule, and give the ranker only that job.
• Write down the point where the plan becomes an action, and put the gate there.
• Make the symbolic layer report the constraint it failed to satisfy, and never swallow it.

A step that needs a guarantee belongs to a planner, a verifier, a controller, or a policy gate. Compose learned judgment and explicit constraints on purpose. Otherwise you ship a system where nobody can name the part that decided.

What Carries

Use language where language is strong. Use external structure where language is weak. Inspect the handoff between them, because by default that boundary has no owner.

Every mistake in this note is a wrong plan on a screen, and a screen has an undo. Fix the handoff while that is still true. The next note puts this loop inside a body, and the undo goes away.

Models solve single steps and lose the thread once the steps compose. A replayable tape buys more reliability than a better prompt. A workplace agent runs across APIs, forms, browsers, tickets, calendars, documents, and business rules. The hard part is holding state across that mess, where one wrong action corrupts the task.

Without a tape, the agent becomes a run of tool calls and confident summaries. It clicks, updates records, and moves on, and nobody can tell afterward whether it helped.

API Agents and Web Agents Fail Differently

API agents act through structured calls. The action space is named and small, and the system knows the schema.

Web agents act through pages. They handle visual layout, hidden state, DOMs that change under them, and buttons whose labels are ambiguous. People built those workflows for people, and nobody rewrote them for a model.

A rejected call reports its own error. A wrong click can leave a page that looks correct and a record that is now wrong.

Enterprises carry both. The APIs exist, and real work still leaks through browser screens. The note on software agents made the interface the part you build. Here you inherit the interface.

TapeAgents Treats the Trajectory as an Object

TapeAgents keeps the trajectory as an object. The tape records thoughts, actions, observations, and decisions in the order they happened. It also holds what the user asked for, what failed, and what is still blocked.

By replayable I mean the tape alone reconstructs the run, with the agent switched off. You debug by replaying and finding the step where it turned. You improve the agent from the same tape, because it is data about its own behavior.

A tape that records only call payloads goes blind where the web agent works. Page observations belong on it too.

The grounding note asked one question of every answer: which shelf did this come from. A workflow adds two more: what did you change, and how does a person undo it. The tape answers all three.

Models Solve Steps and Lose the Composition

WorkArena-style benchmarks measure common knowledge work. The agent must work inside enterprise software, satisfy constraints, and finish a task that takes many steps.

The benchmarks expose a common weakness. Models solve isolated steps and struggle once those steps compose. The failure sits in planning, memory, interface interpretation, and error recovery more often than in missing knowledge.

Each of those is a state problem. Planning needs the goal and the list of finished subtasks. Error recovery needs what the agent already tried and what failed. Both live outside the model.

A demo hides all of this. The person driving supplies the missing state between steps, and the overview note called that demo thinking. In production nobody holds the thread, so the agent must carry it.

State Is the Safety Feature

The system must know what it read, what it changed, and what still needs approval. It must also tell which assumptions came from the user and which came from a tool.

Drop that state and the failures are ordinary. The agent repeats steps, forgets constraints, submits half-finished forms, and treats a stale observation as current truth.

The tape is the file. The four states are its columns, written at every step and readable at any row. Remember is the loop step that fills them.

• Task state: the goal in play, and the subtasks already finished.
• World state: what the agent saw, and when that observation was valid.
• Permission state: which actions need a confirmation or a human approval.
• Evidence state: which sources justify the current answer or action.

Permission state has to sit apart from task state. An agent that infers permission from its own progress approves itself.

Write the workflow as states, transitions, roles, evidence, approvals, deadlines, and exceptions before you assign a model anything. Then give the model a job inside it: see, draft, classify, route, or act. Start with the model and you get generated messages chained through a process. That breaks at the first exception that needs a role, a deadline, or a human decision.

Ask a Human at Decision Boundaries

An agent that asks a precise question is safer than one that guesses. The design job is to make those interruptions rare, specific, and worth the user's attention.

Approval on every step is a failure with a friendly face. The user does the task twice, once by watching and once by clicking. The approvals become a reflex, and the one that mattered goes through unread.

The interrupt fires when the agent spends money, changes records, contacts people, deletes data, or acts on uncertain information. Everything else it does without asking.

That rule lives in the permission column, and the tape records who approved what. A workflow can run end to end and still need approval gates and rollback points.

The Builder Test

Replay one finished task from the tape with another person reading over your shoulder. They must see every observation, action, result, permission state, and approval without asking you a question.

Where they have to ask, the tape is thin at that step. Repair the tape there before you widen what the agent is allowed to touch.

What Carries

Judge workflow quality after the task finishes, by whether a person trusts the path. A finished task proves the agent moved. A replayable tape proves someone can follow every step, resume the task, and undo it.

A tape records what happened. Some tasks need a component that refuses to let the wrong thing happen at all.

The interface is the part of a coding agent that a team builds, and it sets the ceiling on the model inside it. SWE-agent showed the mechanism: better commands and clearer observations improved results while the base model stayed fixed.

Code Has an Oracle

Coding agents are the easiest agents to overrate and the easiest to check. Code has tests. The agent can claim it fixed the bug, and the test suite gets the last word.

Autocomplete suggests and an assistant answers. An agent inspects the repository, edits files, runs commands, reads the failure, and tries again. Each rung takes more responsibility, and only the last one runs the code.

The Agent Sees the Repo Through Its Tools

A coding agent sees the repository through a narrow surface: search, open file, edit, run tests, read output. Everything it knows about the code arrives through that surface. In a real repository the same channel carries linters, type errors, and version-control state. When those tools are clumsy, the agent makes worse decisions with less to go on.

SWE-agent gave that surface a name, the agent-computer interface, and treated it as the variable to move. Command design, observation format, and file localization are edits a team makes without touching a weight.

The loop note made the same argument for a browser agent, where the action space decides how much the model must infer. Code shows it first because a failing test names the line that broke.

A Bad Action Space Gives Too Little Power or Too Much Rope

A bad action space fails in two directions. An agent that reads one terminal line at a time burns its context on noise. Hand it the power to rewrite the repository with no guardrails and the patch comes back too big for anyone to review.

The surface that works copies how a careful engineer already moves. Search symbols, read the nearby code and its style, and make one narrow change. Run the smallest test that applies, and widen the change only when the evidence demands it.

• Search must return ranked, structured context, never a wall of text.
• Edits must be patch-shaped, easy to review and easy to revert.
• Test output must come back summarized, with the failing line kept intact.
• Every command must carry a cost label: cheap, expensive, risky, or destructive.

File Localization Is Half the Battle

Most coding tasks start as a search problem: which file owns the behavior, which function matters, which test describes the failure.

The agent that reads everything loses. The agent that narrows fast wins. Localization is the first engineering act in a run, and every later step inherits it.

The Best Runs Are Boring

A serious coding agent lives inside one cycle: orient, hypothesize, edit, run the test, read the failure, repeat. The best runs feel boring because every step has evidence behind it.

The overview drew that shape as observe, decide, act, examine, remember. A coding agent fills the examine step with a command that returns a verdict.

An agent can jump from the issue text to a confident patch. When it cannot show how it localized the behavior, it skipped the step where most engineering judgment lives.

SWE-bench Still Passes Patches That Miss the Intent

SWE-bench asks agents to fix real GitHub issues, which puts it closer to real work than multiple choice. The scoring stays messy anyway.

Tests flake, issue descriptions arrive incomplete, and dependencies stay hidden. A patch passes the suite, misses the intent, and the benchmark marks it solved.

The reasoning note put the verifier outside the model. A test that passes while the intent fails is a weak verifier. The agent cannot see that gap from inside its own loop.

Read the final report against the diff. That check catches the loud failures: the wrong file changed, unrelated work overwritten, success declared before the code ran. The other check is whether the tests exercised the changed behavior. A green suite that never ran the new path proves nothing.

The Builder Test

Ask what the agent learns after a failed test. Take one run, stop at the first red test, and read what the agent does next.

A loop that cannot turn failure output into a smaller, better patch is not a coding workflow yet. The next patch must be narrower than the last one, and the failing line must explain why.

What Carries

Carry the edit-test loop. A test run is the oracle that tells the agent whether the patch touched reality. The interface decides whether the agent can reach that oracle and read what it says.

Code gets that oracle for the price of a command. Most workplace tasks have no oracle at all. The record of what the agent did is the only thing left to read, so build the record before you need it.

DSPy replaces prompt tweaking with a signature and a metric. The metric you pick becomes the system you get. Treat every model call as a module inside a program. Then optimize the program against a number you chose on purpose.

Real Tasks Need More Than One Call

One model call is convenient and narrow. A real task needs retrieval, decomposition, ranking, verification, tool calls, and retries, and none of that lives inside the call. A compound AI system is what you build around the model to cover the rest.

The compound system can beat the model alone because it adds structure. It decides what context to retrieve, how to ask, how to combine outputs, and how to examine them.

The overview note gave the alternative a name: demo thinking. A person supplies the missing state between steps, and the demo works because of the person. One giant prompt has the same shape, with the state buried in a paragraph nobody can score.

DSPy Turns Prompting Into Programming

DSPy reframes prompting as programming. A signature says what goes in and what must come out. Modules compose into a pipeline. An optimizer then searches for instructions, demonstrations, or parameters that move the metric you chose.

question -> retrieve passages -> draft answer -> verify against passages -> revise -> final answer with citations

Each arrow in that line is a place to measure. Most prompt work never gets that far. You change a sentence, look at the output, and hope it generalizes to the next question.

Define the task, define the metric, run the optimizer, then compare results. Comparison is the step that turns a change into evidence.

The Signature Is the Quiet Win

The signature is the part people skim. It forces you to name what information enters the module and what shape the module must return.

A signature turns "ask the model nicely" into a contract. With a contract you can test the module and swap it. You can also reason about how it composes with the modules on either side.

Ownership starts at the contract. When a run goes wrong, you want one question answered: which module owns the failure? Without named inputs and outputs, the answer is the whole pipeline, which is the same as no answer.

The Optimizer Gives You Exactly What You Measure

Optimization helps when the pipeline and the metric match the real job. Aim the optimizer at a cheap proxy and you get a system that is excellent at the proxy.

Metric overfitting is the standard failure. The optimizer moves whatever the score rewards, and the score climbs while the product gets worse.

• Reward answer length and the optimizer learns verbosity.
• Count citations on the surface and the system learns to cite without understanding.
• Leave latency and cost out of the score and the optimizer can build something impressive and unusable.

Bad tests produce bad software. Bad evals produce bad agents. The reasoning note drew the line: reflection happens inside the model, and verification happens outside it. A metric is that outside signal written down, and a weak metric is reflection with a number attached.

An evaluator decides whether a change improved the system or only made one demo prettier. Someone has to design that evaluator, so judgment moves earlier, into the metric and the module boundaries. That work happens before the first tuning run, and it is the expensive part.

Agents Have Programmable Insides

An agent looks like an open-ended loop from the outside. The inside is a set of modules you can program and score. Retrieval, tool choice, answer synthesis, and verification each take a named input and return a named output.

Teams add prompt steps until the examples look better. Without module-level evidence, the system gets slower and harder to debug while the real bottleneck stays untouched.

Attach examples and a metric at each interface, and a regression gets a return address. Keep the examples that expose regressions, and run them against every tuned version. Then you can show which component changed when the score improved.

The move is to replace vibes with interfaces. A coding agent needs files, tests, commands, and patches. A workflow agent needs state, permissions, rollback, and logs.

The agent-loop note put the rule plainly: the weakest interface sets the ceiling. A compound system makes those interfaces visible enough to rank.

The Builder Test

Take a system you run this week. List its interfaces: the tool schema, the retrieval quality, the state format, the evaluator, and the permission boundary.

Rank them by what a bad version of each costs you. The lowest one sets your ceiling, and prompt work does not raise it. I start at the evaluator, because a broken evaluator hides every other problem on the list.

Then write down which module owned your last bad output. If nobody can name it, name the interfaces before you tune anything else. Read the tuned program afterward. A pipeline no one on the team can follow is a black box you now own.

What Carries

A metric is only as good as the oracle behind it. Code carries the strongest oracle in this unit, because a test suite answers without an opinion. Take your weakest module to a task where the oracle runs itself, and find out whether your metric survives contact.

A larger context window holds more noise as easily as more truth. Grounding shows which shelf the answer came from. That difference separates a demo answer from an answer someone acts on inside a business process.

Enterprise Work Has a Different Failure Budget

A model can be fluent and fast and still be unusable inside a company. One wrong answer that a person acts on costs more than one right answer earns. That asymmetry is the failure budget.

By usable I mean four things. The answer grounds every claim in company data. Access controls bind the agent the way they bind a person. Messy workflows do not break it, and the output leaves a trail an auditor can follow.

That budget changes the question. One impressive run shows the model can do the task once. A company asks whether the system helps it make fewer bad decisions.

Enterprise Work Arrives as Slides, Tickets, and Screenshots

Clean text is rare. Real work shows up as slides, PDFs, spreadsheets, logs, screenshots, emails, charts, database rows, tickets, and meetings.

A useful agent moves across those forms. It reads, compares, extracts, summarizes, and acts. Flattening them into text too early destroys the structure the answer depends on. A spreadsheet turned into a paragraph loses the row and the column that carried the number.

Form decides what a citation can point at. You can cite a row, a cell, or a ticket. A paragraph that was once a table gives a reader nothing to check.

Long Context Is Room, Grounding Is Knowing the Shelf

Long context helps, and it stops short of an answer you can defend. Put more tokens into the prompt and three failures remain. The model misses the one detail that mattered. Two documents disagree and nothing in the system settles which one wins. The answer arrives with no way to show where it came from.

Teams list access control and provenance as two separate requirements, and they are two ends of one path. The permission check decides what enters retrieval. The citation proves what came out.

A document that slips past the permission check contaminates every answer built on it. Nobody can examine an answer that cites nothing. Grounding also means the system refuses to treat stale or unauthorized text as instruction.

The reasoning note drew the same line. Reflection happens inside the model. Verification happens outside it, and a citation is verification a company can run without trusting the model.

Production Rewards Repeatability

Most demos work because a person supplied the missing state, which is the trap the loop note named. Production removes that person, and every gap the person filled becomes a defect. A demo can sound correct and still read the wrong document. It can ignore permissions, miss a newer record, or take an action nobody can audit.

When I evaluate an enterprise agent, I ignore the first beautiful answer. I watch the tenth messy run and check five things.

• Give the agent only the documents and tools this user can open.
• Make every claim that matters point back to a source, a row, a ticket, or a tool result.
• Name the one place the system changes when the business process changes.
• Decide which tasks you measure offline, which you watch live, and what failure rate you accept.
• Route low confidence to a pause, a question, a human, or a refusal.

Irreversible actions get their own gate. An agent must pause before any step that can cost money or trust, and a person must approve that step.

The Intern With a Badge

A new intern reads fast and writes well. Nobody hands that intern every file cabinet, every customer record, and the power to approve invoices on day one. The intern gets a badge, a checklist, a manager, and a logbook.

Enterprise AI has the same shape. The workplace decides what the agent can see and what the agent can do. It also decides how everyone else inspects the result.

A conversation about productivity skips the parts that decide the outcome. It leaves out data access, permissions, compliance records, and the case where the agent is persuasive and wrong. Walk one task through the control surface instead.

• Record which identity performs each action.
• List the data that identity can open, and stop retrieval at that boundary.
• Assign one person to review exceptions.
• Pick the business metric that proves the workflow improved.

An enterprise agent earns trust when it becomes an accountable participant in the control systems the company already runs.

The Builder Test

Take one answer your agent produced and walk it backward to evidence. Write down which records the permission check allowed, which of those retrieval returned, and which ones the answer cited. Find the place where two sources disagreed and say what settled it. Name the action that waited for a human approval.

Any step you cannot fill in is a step the system does not have.

What Carries

A user must be able to ask what the agent relied on, what it changed, and how to undo it. An agent that cannot answer all three is not ready for a workplace. Those three answers are the product.

Grounding is a promise about evidence. The next note makes that promise scoreable.

A framework earns its keep when it makes a hidden decision explicit: who owns state, who calls tools, and what ends the loop. Those decisions exist whether or not the framework names them. The coordination machine is the part your team writes. It decides who talks, what state the run keeps, which tool the system calls, when the loop stops, and how a person takes control.

A Second Agent Must Remove a Bottleneck

The multi-agent pitch is a room of specialists. Planner, coder, critic, researcher, executor. Each has a role, they pass messages, and the work improves.

Sometimes it does. Sometimes you get a meeting with no manager. The value comes from structure. That structure has clear roles, bounded turns, explicit handoffs, shared state, and a test that decides when the result is good enough.

The loop note named five steps: observe, decide, act, examine, remember. A role earns its existence by owning one. The planner owns decide, the executor owns act, and the verifier owns examine. A system where all three do the same job has one agent and three labels.

Add a second agent only when it removes a named bottleneck: different tools, different context, different verification, or different ownership. An agent that shares all four only adds turns.

AutoGen Turns Conversation Into Control Flow

AutoGen made the multi-agent idea concrete by treating conversation as a runtime primitive. You give each agent a role, connect them in a group chat, hand them tools, and set a stop condition. Control flow becomes the question of who speaks next.

The gain is separation. One monolithic chain runs a single voice through a single context. A group chat runs specialized participants under rules, so the research context stays out of the coding prompt.

The cost lands on state. In a group chat, the shared state is the transcript. The ReAct note asked you to replay one run from logs and explain every transition, and a transcript does not survive that test. You can read what each agent said, and you cannot recover which fact the system relied on when it acted.

StateFlow Trades Flexibility for Inspectability

Free-form conversation is flexible and hard to control. StateFlow puts the run in a named state, and it allows only the transitions on the list. The run stops for a reason the system can print.

You give up the move where an agent improvises a new step mid-run. You get four answers instead: the current state, the legal transitions, the evidence that moved the task forward, and the reason the run stopped.

The state machine is less magical than a group chat, and in production that trade pays. A named state and a stated stop reason let a colleague read a failed run they never watched.

LlamaIndex Is the Catalog the Agents Read From

An agent reads from whatever it can reach. It needs data, and data needs structure before retrieval works, so without the catalog and the labels the assistants wander.

A useful RAG system is a pipeline: ingestion, parsing, indexing, retrieval, reranking, synthesis, citations, and evaluation. The language model is one stage in that list. Multimodal knowledge assistants run on the same stages, and parsing turns mixed documents into something an index can hold.

Citations are the stage that decides trust. The test is whether the system can name the source of each retrieved fact. A system that retrieves well and cites nothing gives you a confident answer you cannot check.

A Framework Is Worth Its Weight When It Names the Contract

Write the contract before you pick the tool. The contract names the owner of each step and the tools each role can call. It also names the context the roles share, the memory the run writes, and the condition that counts as done.

With the contract on paper, the choice of framework stops being taste. Match the mechanism to the part of the work that needs it.

• Conversation fits the work where roles must negotiate a plan.
• A state machine holds a workflow where correctness outranks flexibility.
• Retrieval belongs wherever an answer must rest on evidence.
• An evaluator turns a run into a score you can improve against.

Adding a framework layer does not make state clearer by itself. More agents mean more confusion when nobody owns the facts, the tools, the errors, and the final decision.

The Builder Test

Take a system you run now and draw its contract on one page.

• Name the owner of state, then check that a second engineer can find it without reading the transcript.
• List the tools each role can call, and mark the ones whose actions are expensive to undo.
• Name the evidence the run saves, and make sure each retrieved fact carries its source.
• Write the stop condition, and make it stronger than the model announcing that it is done.
• Name what sends the task to a human, and name the human.

A field you cannot fill is a decision the framework already made for you. Read the framework code until you find where that decision lives.

What Carries

A framework gives you places to define reliability: schemas, permissions, retries, evaluators, logs, and handoff rules. Filling them is your job.

The lazy choice picks whatever makes the first demo shortest. The bill arrives later, when hidden memory, implicit retries, and an unreadable trace make one production failure impossible to explain.

A contract inside one team is a design choice. A contract inside a company is a permission boundary.

A model without tools must guess what it does not know. A model with tools can look, calculate, retrieve, run, and adapt. ReAct alternates the two in one loop: think, act, read the result, think again. The action space decides how much guessing is left.

The overview note drew the agent as five steps: observe, decide, act, examine, remember. The reasoning note bought the model room to think and left the check outside. Act is the step that reaches the check.

An Agent Interacts With an Environment

An agent acts inside an environment and reads what comes back. The environment is physical for a robot arm and digital for a browser or a terminal. It is human when the user holds context the agent has no way to fetch.

The human case carries state no tool exposes. A user who answers one question can end a search the agent cannot finish alone.

A language model agent uses text as the interface for all three. It reads observations in text, writes thoughts in text, emits actions in text, and receives the next observation in text. One channel carries the whole loop. A browser, a shell, and a person sit behind the same interface.

ReAct Alternates Reasoning With Action

The reasoning step keeps the search purposeful. The action step gathers what the model does not hold. Alternating them means each plan gets revised against something that came back from outside.

Thought: I need the current fact, not a memorized answer.
Action: search(query)
Observation: new evidence arrives
Thought: Now update the plan
Action: call_tool(args)
Observation: tool result
Final: answer with evidence

The feedback does the work. A trace can read as coherent while the action it justifies is wrong. The observation is the only part of the turn the model did not write.

Memory Changes What the Loop Can Learn

A context window is short-term memory. It works inside one run and disappears with it. Long-term memory keeps lessons, facts, plans, failures, and skills outside the prompt, then returns them later.

Voyager stores the code skills its agent learned in Minecraft. Later runs start from earlier solutions. Generative agents let memory logs shape what the agent does next.

A chatbot is a student with a whiteboard wiped at the end of every class. An agent with memory is a student with a notebook. The notebook adds no intelligence by itself, and it changes what learning can mean.

The Action Space Decides How Much the Model Must Infer

Tool use is more than handing the model an API. The action space fixes what the model sees, what it must infer, and what a bad move costs.

A web agent that clicks one page at a time drowns in tiny observations. An interface that summarizes candidates, exposes page structure, or searches directly leaves less to infer.

Grain is a recovery decision. Small actions keep every mistake cheap and every task long. One large action finishes sooner and takes more of the run down with it. Pick the grain by what an undo costs.

The model sets the floor for a task. The interface it acts through sets the ceiling.

The Loop Needs State

Memory is what survives the run. State is what survives the step. An agent without state improvises every turn from scratch.

It can answer well in a short exchange. Long tasks break, because nothing durable holds the goals, decisions, observations, tool results, and constraints.

State does not need to be fancy. A task list, a scratchpad, or a database row counts. So does browser state, a file diff, a memory record, or an event log. The next action comes from what happened, not from what the model half remembers.

Replay a failing run step by step. Read the observation, the prompt assembly, the action proposal, the permission check, and the tool result. Then read the state update, the retry, the stop, and the report. You must explain every transition from what the logs recorded.

Blaming the model is the brittle habit. The real defect is usually an absent transition rule or stale context. A weak stop condition or an unchecked tool result does the same.

Where the Loop Breaks Today

Four weaknesses stay open, and none of them live in the prompt.

• Training: most base models never saw a long agent rollout.
• Consistency: a real task needs the same result across many runs, and one lucky pass proves nothing.
• Human collaboration: many tasks need a clarifying turn before autonomy.
• Benchmarking: single-shot accuracy misses multi-turn failure modes.

Pick the one that will bite your task first and instrument it before the next run.

The Builder Test

Trace one task turn by turn. If you cannot say what the state holds after each action, the loop is not observable enough.

• Name the transition that created the bad state.
• Separate the retry that is safe from the retry that repeats the error.
• Say whether stop meant success or only the absence of a next action.

A transition you cannot explain from evidence is where the next bad state will start.

What Carries

The loop is the unit of reliability. An agent is a repeated interaction with an environment, and every turn writes what the next turn reads. The interface decides what that record can hold.

One loop with one owner is the simple case. Add a second agent and the first question is who owns the state.

Chain-of-thought buys the model room to work. Only a check outside the model turns that room into a correct answer.

Before an agent touches a tool, it has to work through a hard task without drifting. That is the oldest problem in AI, and tool wiring does not skip it.

The map note draws the loop in five steps: observe, decide, act, examine, remember. Reasoning improves the decide step, and it leaves the other four alone.

Intermediate Steps Change the Continuation

A language model is trained to produce a plausible continuation. Truth is not the training target. Intermediate steps change what a plausible continuation has to contain. The model builds the answer across many tokens rather than one.

The steps are a scaffold. They give the model somewhere to allocate attention and hold partial results.

Last-letter concatenation shows the flip. Ask for the answer and the model treats the task as a pattern over names. Show the decomposition and the same model runs a sequence of tiny operations. The task goes from brittle to reliable.

Self-Consistency Is a Vote Over Paths

Sample many reasoning paths instead of one, then pull the final answer out of each. Let the answers vote and take the winner.

A single path fails for local reasons. One early token that misleads drags the rest of the solution with it. Many samples give you several attempts at the same latent process, and agreement across them raises confidence.

Ten kids run a maze and seven come out the same door. That door is probably the exit. The vote carries no guarantee, and it still beats the first kid who ran.

Analogical Prompting Asks the Model to Build Its Own Examples

Analogical prompting hands the example writing to the model. Ask it to recall or invent related problems, solve those first, then carry the shape to the real one.

Hand-written examples cost you time, and each one covers a single shape. A tutor picks one close cousin, names the shape, and lets the student carry it to the rest.

Correction Needs a Signal From Outside the Model

Self-correction is no safety net. Ask a model to review its own answer and it can repair a wrong one. It can also break a right one. An instruction to check your work does not create ground truth.

A trace creates intermediate state, and that state is not automatically correct. A fluent chain can drift or carry a wrong early assumption all the way to the answer. The signals that catch it are cheap and specific.

• Unit tests for code
• A calculator for arithmetic
• A symbolic solver for constraints and a verifier for proofs
• Retrieval for facts
• Environment feedback for plans
• A human for judgment

When a reasoning step can change an action, you must be able to examine that step.

Later notes build these signals into the loop. The software note uses tests, and the neural-symbolic note uses a solver. The scaling-policy note carries the same distinction to the release gate.

Agents Multiply Reasoning Mistakes

A wrong answer is one bad output. A wrong plan calls tools, writes files, spends money, emails people, and pushes a bad decision downstream.

The fix is a bound on what a wrong chain can touch. Four moves put that bound in place.

• Trace the run so each step is visible
• Sample so that no single path gets your trust
• Run the verifier before the action
• Stop the run when the verifier fails

An agent that keeps going past a failed check does not have a check.

The Builder Test

Take one task your agent gets wrong today. Add the cheapest check that lives outside the model, a test or a calculator. Run the same set of tasks both ways and count what the check caught.

Compare five shapes on the same tasks: short, decomposed, tool-verified, search-guided, and one with the scratch work hidden. Score every shape on the rate of errors that matter.

Read each intermediate step on its own. A step you cannot examine alone is a step you are trusting. If you cannot say what changed after each step, the run is not observable enough to test.

Check that your decomposition matches the real structure of the task. If a verifier catches more of those errors than a longer answer, ship the verifier.

What Carries

A reasoning trace earns its cost when it improves a decision or exposes something you can examine. Length proves nothing on its own. A long explanation can hide anchoring, circular justification, and an early mistake the chain preserves.

A verifier lives outside the model, so the agent needs a way to reach it. That is the act step.

An agent is a language model inside a loop, and the loop is the part a team builds. The model call is one node in it. What the loop sees, what it can touch, and what it saves are engineering decisions. So is the rule that stops it.

You meet an agent as a product surface first. A chat box gets a tool, the demo runs, and the room calls the result magic.

The course treats the agent as a system. The model sits in a loop, wired to tools and placed in an environment. Tasks measure it, and safety constrains it.

The Model Is One Call Inside a Process

The loop has five steps: observe, decide, act, examine, remember. The model call sits at decide. Tools make act possible, the environment supplies observe, and storage gives remember something to hold.

The process decides what the model sees, what it can touch, and how it recovers. It also decides how anyone knows the run worked. Draw the boundary around all of that and you have an agent. The model call alone is a text generator with good manners.

A complete agent has parts you can point at: loop state, tool contracts, memory, permissions, observability, evaluation, escalation, and rollback. Leave one out and the system still demos well. It fails on the first run where the missing part is the one that mattered.

Reliability Is the Standard

Reliability is the standard, and it is a harder bar than a good answer. Reasoning counts when it survives a distraction. Tools count when the interface is shaped well enough that the model does not have to guess. Memory counts when retrieval returns the right record at the right moment. A second agent helps when coordination beats confusion.

The standard is what still matters when you build with it next week. Four checks make it concrete.

• Name the state that persists between actions.
• List the actions that can change the outside world.
• Point at the evidence that ends the task.
• Show how a human takes control back.

Run the four before you widen what the agent can touch.

The Course Moves From the Inside Out

The twelve notes move from the inside out, and each one repairs a joint of the loop.

• Notes 1 and 2 build the decide and act steps. Reasoning traces make the decision less brittle. ReAct and tools ground the action, and memory makes the next observation less blind.
• Notes 3 through 7 make the loop inspectable: multi-agent frameworks, enterprise grounding, DSPy, coding agents, and workflow agents. They settle who owns state, what counts as evidence, and how a run gets measured.
• Notes 8 and 9 take the work language cannot hold: symbolic planners, solvers, simulation, robotics, and embodied control. Exact constraints move into the decide step, and the act step moves into a physical world.
• Notes 10 through 12 decide where the loop stops. Open science, measurement, responsible scaling, privacy, and trustworthy deployment draw that line.

Each group assumes the loop underneath it already exists.

A Demo Works Because a Human Filled the Gaps

A demo works once because a person fills the gaps by hand. You supply the state the loop never stored and clean the tool output before the model reads it. When the recovery path leads nowhere, you step past it.

None of that work is in the system. The next run happens without you in the room, and that run is the measurement.

Autonomy pays when the agent can see state, act safely, measure progress, keep evidence, and stop before mistakes compound. Without those five, more autonomy moves a bad decision further before anyone catches it.

Ask Which Part of the Loop Failed

When an agent fails, name the step that broke before you touch the prompt. Observe, decide, act, examine, remember, and the permission boundary are the candidates. A prompt edit moves the decide step and leaves the other five where they were.

Read the transcript and you see one run. Read the machinery and you see every run after it. Then check whether that machinery holds under noisy inputs, stale state, weak tools, and partial failure.

Fix the named step, run the task again, and check the same six candidates.

The Builder Test

Take one task your agent runs today and trace it through the loop. Point at the observation it started from and the action it took. Find the result, the memory it wrote, and the measurement that says the task is done. Name the boundary that held it inside its permissions. A step you cannot point at is the defect, and it is the next thing to build.

What Carries

Agent quality is state, tools, evidence, measurement, and control working together. Model quality is one input to that. The other five are yours to build.

A loop that cannot think through one hard task will not survive twelve. So the decide step comes first.