# From Copilot to Colleague
> How AI Engineering Turns Models into Dependable Systems

By Timur Isachenko. Built from 794 AI Engineer talks · 54 claims · 199 source anchors.
The complete book as markdown. Source: https://fromcopilottocolleague.com · https://github.com/isatimur/ai-engineering-book-lab

# Chapter 1 — The Shift: From Assistant to Delegate

The most important change in applied AI is not that chat got better. It is that people stopped wanting an answer and started wanting the work done.

For a few years the dominant experience of AI was answer production. You asked, and the system summarized, explained, drafted, brainstormed — with a fluency that mattered. But it was an interface story. A smarter text box. A faster first pass. A more conversational way to reach the software you already had. The verb was always *tell me*.

The sharper change begins when the verb becomes *go do*. Research this topic. Draft the contract. Refactor the service. Triage the queue. Investigate the failure. Come back with something a person can actually review and use. The moment the request crosses from "tell me" to "go do," the standard for success changes underneath it — and most of this book lives in the gap that opens up.

## Three words for three different relationships

It helps to be precise about what is shifting, because the words get used loosely. The cleanest test is where the human sits relative to the loop: in the critical path for every step, alongside the work in real time, or out of the loop until review. Assistant, copilot, and delegate are those three positions, and moving between them is not the same system scaled up — it changes what the system has to be.

An **assistant** suggests. It produces something you then decide what to do with — a draft, an answer, an option. The human stays in the critical path for every step, and the assistant's job is to make that step faster.

A **copilot** collaborates inside a human loop. It works alongside you in real time, and the canonical case is the coding copilot completing the line you were already typing. The human is still flying the plane; the copilot is making continuous small contributions to a task the human is actively driving.

A **delegate** is assigned work and expected to return with it done. Not a suggestion to evaluate, but an artifact, a recommendation, or a completed step. The human steps *out* of the moment-to-moment loop and re-enters at review. That single move — out of the loop, back at the end — is the whole shift, and it changes what the system has to be.

This book's claim is not that assistants and copilots disappear. They remain useful, and for many tasks they are the right tool. The claim is that the engineering difficulty, the product ambition, and the organizational upheaval have all migrated to the third category. Delegation is where the hard problems are, because a delegate is no longer just saying things. It is shaping work that someone else will rely on.

## Why delegation changes the failure surface

When an assistant is wrong, the cost is bounded by the fact that a human is reading every word it produces. The error surfaces immediately, in context, with the person who asked still holding full attention. Suggestion is safe partly because it is supervised by construction.

Delegation removes that built-in supervision. The human is not watching each step; they are waiting for a result. So an error no longer surfaces when it happens — it surfaces later, downstream, possibly after it has been built on. A wrong suggestion is a wrong sentence. A wrong delegated action is a wrong artifact that other work now depends on. The failure surface stretches from a single response across an entire workflow, and that stretch is exactly what makes delegation an engineering problem rather than a UX one.

This is why so many practitioners describe the same wall. Jacob Lauritzen, building legal AI at Legora, puts it plainly: vertical AI and complex agents "need more than just the chat." The chat was never the hard part. The hard part is everything that has to exist around it before the work it produces can be trusted without someone watching it happen.

And the gap is not closed by a smarter model alone. Barry Zhang and Mahesh Murag at Anthropic name the distinction carefully: agents "have intelligence and capabilities, but not always expertise that we need for real work." Their illustration is exact — you would not ask a 300-IQ mathematician to derive the 2025 tax code from first principles when what you need is an experienced tax professional's consistent execution. Capability and expertise are different things, and the practical move is to supply the second deliberately rather than wait for the first to grow into it: package the missing context, conventions, and procedures as reusable skills the agent loads, which is precisely the remedy Anthropic reaches for. Intelligence is necessary. On their account it has not, by itself, turned out to be sufficient.

## The chat box was always the smallest part

Once a system is doing real work rather than answering questions, the conversational surface becomes only the visible tip of it. The actual product is the apparatus underneath, and it is worth treating that apparatus as a checklist when you scope a delegation feature: context assembly that decides what the system knows, tool access that decides what it can do, workflow structure that holds a multi-step task together, quality checks that catch errors before a human does, durable state that survives interruption, review layers where a person re-enters, and observability so that person can see what happened. If a delegated workflow is missing one of these, that is where it will fail first — and the chat box is the one part you can take for granted.

This is why "agent" products keep sprouting the same organs. They grow trace views, side panels, memory layers, approval queues, workflow diagrams. None of that is decoration; read it as a diagnostic instead. When a serious agent product lacks a trace view or an approval queue, treat the absence as a reliability gap, not a leaner design — it means the trust those organs externalize is still implicit, riding on a human watching. They are the reliability work becoming visible. Joel Hron at Thomson Reuters describes the target as systems that don’t just suggest but plan their own work, execute it, and replan as they learn. Every word past *suggest* in that sentence is a new engineering surface, and the rest of this book is largely a tour of them.

The evidence for this shift is not a controlled study proving that broad delegation already works well everywhere. It does not yet. What the corpus shows is something more specific and, in its way, more credible: a strong convergence among serious builders about what they are *trying* to make these systems do, and a remarkably consistent account of where it gets hard. This book reports that convergence and the engineering it is producing. It does not claim the problem is solved.

## Two cases this book keeps returning to

Both cases stress the same idea from opposite ends — one in software, one in knowledge work — and the use of pairing them is diagnostic: when each independently points past the model to the scaffolding around it, that is the signal the scaffolding is the real variable, not the model.

The first is the **software factory**. A coding agent looks magical on a small, self-contained task and then degrades on larger ones. The trap is to read that degradation as a model ceiling and wait for a better model; the move practitioners keep making instead is to improve the workplace around the agent — the repository, the harness, the specs, the evals, and the runtime. When delegated work falls apart at scale, fix the environment before you blame the model: raw capability is not enough for dependable delegated work, and the workplace itself has to be made legible to the agent. Chapter 3 takes this up directly.

The second is the **high-stakes colleague**. A legal, tax, compliance, or enterprise-research assistant begins as a helpful conversational surface and then gets asked to do work with professional consequences. At that point provenance, access boundaries, retrieval discipline, durable trajectories, and explicit review points stop being optional. Fluency is still useful, but it is no longer the thing being bought. Chapters 5 and 7 return here.

The two cases are deliberately unlike each other. One lives in software, where artifacts are executable and testable and a failing test is an honest signal. The other lives in knowledge work, where authority, evidence, and institutional accountability carry the weight and the failure is a wrong judgment rather than a red build. They are different enough that their agreement means something. Both arrive at the same conclusion: once the task becomes delegated work, intelligence alone is not enough.

## What this opening sets up

That conclusion hands directly to the next chapter. If delegation makes execution cheap, it does not make judgment cheap — it makes judgment *scarce*, and therefore more valuable. Taste, standards, and review quality stop being background virtues and become the constraints that keep cheap output from turning into expensive mess. Chapter 2 is the human half of this opening argument, and it has to come before the technical core, because the technical core only matters in service of judgment that someone still has to supply.

From there the book asks its real question: what has to surround model intelligence before a team can trust it with delegated work? The answer is a stack of enabling conditions, and the chapters are that stack — stronger human standards (Chapter 2), legible harnesses (Chapter 3), evals as a control system (Chapter 4), context as infrastructure (Chapter 5), durable runtimes and human control (Chapter 6), security and bounded authority (Chapter 7), the realtime stress test (Chapter 8), the organization that holds it all (Chapter 9), and finally what survives when the tools turn over (Chapter 10).

So the important fact about modern AI is not that it can talk. It is that people increasingly want it to *work* — not merely to generate ideas but to return artifacts, not merely to answer but to complete bounded steps, not merely to sound plausible but to produce work a human can inspect, redirect, and trust. That demand is what turns model progress into an engineering discipline. The shift from assistant to delegate is where this book begins because it is where the engineering begins.

## What to do with this

- Classify each AI feature you ship as assistant, copilot, or delegate before you build it. Assistant and copilot keep a human in the critical path every step; only a delegate has the human step out of the loop and re-enter at review. If you are building a delegate, expect the engineering, product, and organizational difficulty to be categorically higher — that is where the hard problems are.
- When you cross a feature from "tell me" to "go do," map where errors will surface. A wrong suggestion is a wrong sentence caught in context; a wrong delegated action is a wrong artifact that later work depends on. Before delegating, decide how that downstream error gets caught, because the built-in supervision of someone reading every word is gone.
- Stop expecting a smarter model alone to close the gap. Per Anthropic's Barry Zhang and Mahesh Murag, agents have "intelligence and capabilities, but not always the expertise we need for real work" — so budget explicitly for the missing expertise: the specific context, conventions, and institutional knowledge that turn a plausible result into a usable one.
- Treat the apparatus as a build checklist, not the chat box. For any delegated workflow, account for context assembly, tool access, workflow structure, quality checks, durable state, review layers, and observability — and when a serious agent product is missing one of these organs (a trace view, an approval queue), read the absence as a reliability gap rather than a cleaner design.
- When a coding agent degrades on larger tasks, fix the environment before blaming the model. Improve the repository, harness, specs, evals, and runtime so the workplace is legible to the agent — and apply the same lesson to high-stakes knowledge work, where provenance, access boundaries, retrieval discipline, durable trajectories, and explicit review points stop being optional once the output carries professional consequences.

---

# Chapter 2 — Taste Still Matters When Code Gets Cheap

Cheap code is easy to misread. When a model can generate a feature, a test suite, and the glue between them in the time it takes to describe them, the obvious conclusion is that engineering itself has gotten cheap. Part of it has. Routine production — the typing, the boilerplate, the third CRUD endpoint that looks like the first two — is easier than it was.

But cheap output is not the same as cheap judgment, and the gap between those two is the subject of this chapter. If Chapter 1 argued that the real shift is from suggestion to delegated work, the immediate human question is what remains scarce once the output is abundant. The answer is taste — and abundance does not retire taste. It promotes it.

## The argument in three lines

Generation got cheaper; bad decisions did not. A wrong design choice costs the same to live with whether the code took a week to write or thirty seconds — and a model that ships the wrong thing faster just gets you there sooner. So the skill that decides *what* to build, and rejects what shouldn't ship, is the part that did not get automated, which means it now binds the whole result. Run the test on your own last sprint: if the bottleneck was typing the code, the model helps a lot; if the bottleneck was knowing which code was worth typing, the model changes almost nothing.

A machine can accelerate production without removing the need for discrimination. In fact it sharpens that need, because it widens the gap between what can be produced and what should be. When a team could only build a little, the act of building was itself a filter — you thought hard about what was worth the effort. Remove the cost of building and you remove the filter. Something has to replace it, and the replacement is a deliberate one: a written success criterion *before* the model runs, and a reviewer with veto power *after* it does. Those two checkpoints are the artificial filter you install once the natural one — scarcity — is gone. A useful trip-wire that the filter is missing: the model produced something fluent and working, and no one on the team can state what would have made that same output unacceptable. If you can't name the failing condition, you weren't filtering; you were accepting whatever came back.

Matt Pocock, whose work on software fundamentals runs directly counter to the automation-triumphalist mood, states the consequence bluntly: "Software fundamentals matter now more than they actually ever have." That sounds paradoxical until you see the mechanism. Fundamentals are how you tell good output from convincing output, and a world flooded with convincing output needs that skill more, not less. The model raised the volume of plausible work. It did not raise anyone's ability to evaluate it — that still has to come from a person who knows what good looks like.

## What "taste" actually means here

Taste, in this chapter, is not aesthetics or personal preference. It is the practical ability to tell good output from merely convincing output, which is exactly what Pocock's "software fundamentals" do — and the two kinds of output look identical until you apply them. Make the skill concrete by turning it into questions you ask of a model's diff before you accept it: will this data structure hold under the second use case, or only the one in front of you? Is the error path actually handled, or just absent? Does this function belong in this layer, or did it land here because that was the easiest place to put it? Each question separates local correctness — this code runs — from coherence — this code is the right code. None of those questions got easier when generation got cheap; the model can produce a hundred plausible answers and still not tell you which one passes.

The distinction that matters most in practice is the one between a passing test and a trustworthy result. A model will happily produce code that is green on every check you wrote and still encodes the wrong model of the problem — correct-but-wrong, code you have to rewrite even though nothing failed. Detecting that is not automatable, because the test suite only knows the cases you thought to specify. So the review move is to read past the green checkmark: trace one path the tests don't cover and ask whether the design would survive it. The output that "merely works" and the output that "fits the system" diverge exactly there, at the case no one wrote a test for yet.

These distinctions were always part of senior judgment, but they used to come free. When you wrote every line, you understood it because you made it — comprehension was a byproduct of authorship. A model breaks that bundle: it hands you working code you did not build and therefore do not yet understand, and the understanding now has to be paid for separately, after the fact. The concrete consequence is that "read the diff" stops being a courtesy and becomes the load-bearing step — and reading it like an author rather than a skimmer means reconstructing *why* each choice was made, not just confirming the lines parse. Tuomas Artman, reflecting with Gergely Orosz on craft at Linear, names the open question: "What happens when agents are capable of doing everything immediately for you?" The answer this chapter gives is operational, not philosophical: budget time for the comprehension the model no longer gives you for free, because the scarce contribution moves from *making* to *discerning*, and discerning takes minutes that velocity dashboards do not show.

## Vibe coding: powerful, and dangerous in the same breath

The clearest live example of the taste problem is the practice that has come to be called vibe coding — building by rapid, loosely-specified prompting, steering on feel rather than on a plan. The corpus is split on it, and the split resolves into a single failure mode you can name. Corey Gallon at Rexmore calls it the vibe-coding *hangover*: the app works on Friday, and "then Monday rolls around, you want to add a feature, and you realize that you don't understand it, you can't maintain it, and you have to throw most or all of it away." That is the diagnostic. Vibe coding is safe exactly until the first time the artifact has to change and the person changing it is not the person who prompted it into existence. Watch for the Monday gap — the moment maintenance is required and comprehension is absent — and treat its arrival as the signal that the mode has to switch.

Vibe coding is excellent for exploration. For rough prototypes, interface sketches, internal tools, one-off automation, and the early work of figuring out what is even worth building, it is a remarkable accelerant. Steering on feel is the right mode when the goal is discovery, because in discovery you do not yet know enough to specify, and premature rigor would just slow the learning.

It becomes dangerous at one specific moment: when an exploratory mode quietly hardens into a production philosophy. The problem is not speed. The problem is shipping output that no one fully understands, can maintain, or can confidently review — output that works today and becomes a liability the first time it has to change. Chris Kelly at Augment Code draws the line at the load: "vibes don't cut" production, where production means four-nines uptime, thousands of users, and gigabytes of data — the moment a mistake is paid by people who never saw the prompt. swyx, who runs the AI Engineer community, captures the engineer's reaction to that handoff: "I'm declaring war on slop today." Slop is the precise failure mode — work that looks finished and pushes its real cost downstream to whoever has to understand it next. The practical tell is concrete: if the code path now serves real users or real money, the bar is no longer "does it run" but "would I sign the review," and feel-based steering cannot answer that.

So the right conclusion is not "never vibe code." It is a mode switch with a concrete test: vibe coding is the wrong default the moment the output has to be maintained, reviewed, or changed by someone other than the person who prompted it. Use it freely for the discovery cases named above — prototypes, interface sketches, internal tools, one-off automation, figuring out what is worth building. The skill is not picking a side. It is knowing which mode you are in, and noticing when you have drifted from one into the other without deciding to — typically when an exploratory prototype quietly starts being treated as the thing you ship.

## The friction that was load-bearing

AI removes a great deal of wasteful friction, and that is good. Nobody should mourn the boilerplate. But not all friction is waste, and the dangerous move is to treat every pause as inefficiency to be optimized away.

Some friction is where judgment was happening. The review before the merge. The extra question about whether the architecture can carry this. The refusal to accept a generic draft that technically satisfies the request. The decision to rewrite something correct-but-wrong — code that passes every test and still encodes the wrong model of the problem. Those pauses look like drag on a velocity dashboard. They are often the exact points where quality was being created, and removing them does not make the work faster so much as make the absence of judgment invisible until later. Armin Ronacher, in a talk with Cristina Poncela Cubeiro titled *The Friction Is Your Judgment*, names this directly: the friction worth keeping is "intentionally designed to put" back into engineering workflows — because the alternative is to optimize away the place where the judgment lives.

This is the trap that makes cheap execution treacherous rather than simply beneficial: a badly framed task handed to a strong model wastes more time than it used to, because the system will sprint confidently in the wrong direction and produce a great deal of polished output before anyone notices it was the wrong direction. There is a simple test for which friction to keep and which to cut: cut the pause if removing it only costs keystrokes — boilerplate, glue, scaffolding the model can regenerate on demand. Keep the pause if removing it costs a *decision* — whether the architecture can carry this, whether the draft is right or merely plausible, whether the frame is correct before the model executes it. The first kind of friction is waste. The second is where the judgment lives, and the cost of skipping it now scales with how fast the model executes the wrong frame. Speed without judgment is not neutral; it manufactures expensive mistakes efficiently.

## The new scarce skills: framing and review

Once execution is cheap, two specific capacities become the bottleneck: framing the task before the model runs, and reviewing the output after. Sean Grove at OpenAI states the first half directly — "the new scarce skill is writing specifications that fully capture the intent" — and the rest of this section is what each of those two skills looks like as a concrete practice rather than a virtue.

The first is problem framing. When the model will faithfully execute whatever it is pointed at, pointing it becomes the high-leverage act. Before handing a task to a model, answer five questions explicitly: What is the actual task? What counts as success? Which constraints are real and which are habit? What is allowed to stay rough? What would make the result unacceptable even if it looked finished? Treat that last question as a gate — if you cannot state what would make fluent, working output unacceptable, you are not ready to delegate the task. A team that can answer those questions sharply gets a force multiplier; a team that cannot gets fluent output aimed at the wrong target. Framing is no longer the cheap part of the work before the real work starts. In a world of cheap execution, framing *is* the work.

The second is review as anti-slop discipline. Chris Kelly at Augment Code puts it bluntly: "code review is by far the most important skill," and one the industry under-trained because it interviewed for solving leetcode rather than for reading someone else's code and judging why it is good or bad. That is the skill agents now demand at volume, because every line they write is a line you didn't and therefore have to evaluate cold. The practice has a concrete shape. Itamar Friedman at Qodo names it "vibe coding with confidence," and the confidence does not come from the vibes — it comes from the verification wrapped around them: high-quality tests and a reviewing pass on the generated diff before it merges. So review is not a bureaucratic gate; it is where standards are defended against fast, plausible output, and the operational rule is that cheap output which creates expensive cleanup was never cheap. In an AI-native team the reviewer is not slowing the work down. They are the reason it can be trusted to go fast — the same control-system logic the book applies to evals in Chapter 4, arriving here first in its human form.

## The bridge: from judgment to harness

Framing and review still locate the judgment inside one person's head, surfacing at prompt time and review time. That has a ceiling: it does not scale past the senior engineer's attention, and it spends the model's speed re-checking work after the fact. The next move is to push the same judgment into places the agent reads while it works — the spec it executes against, the validation rules that fail the build, the repository conventions it imitates, the review checklist a second agent runs. That is what Grove's scarce-skill claim points at: a specification is judgment written down where a machine can act on it *before* it produces the wrong thing, not a person catching the wrong thing after.

Externalizing judgment this way — upstream, where it shapes output as it is produced rather than catching it afterward — is exactly what Chapter 3 means by a harness, and it is why this chapter sits where it does. The harness only earns its complexity once you accept that there is judgment worth encoding into it; a team that has not decided what "unacceptable output" means cannot write a validation rule that enforces it.

So when code gets cheaper, the real question was never whether humans matter less. It is which human contribution becomes scarce — and this chapter has named it concretely, not as an abstraction. Taste here is four specific moves: writing the success criterion and the failing condition before you delegate; switching out of vibe-coding mode the moment the artifact has to be maintained by someone who didn't prompt it; keeping the friction that costs a decision and cutting only the friction that costs keystrokes; and reading the model's diff like an author, against the case no test covers. Each is a thing to do on Monday, not a virtue to admire. The rest of the book is the engineering that gives those four moves somewhere durable to live, so they don't depend on one tired reviewer remembering to make them.

## What to do with this

- Before delegating a task to a model, answer the five framing questions out loud or in writing: what the actual task is, what counts as success, which constraints are real versus habitual, what is allowed to stay rough, and — as a gate — what would make the result unacceptable even if it looked finished. If you can't state that last one, you're not ready to delegate yet.
- Decide which mode you're in before you start, and re-check when output is about to ship. Vibe coding is the right default for prototypes, interface sketches, internal tools, one-off automation, and figuring out what's worth building; it's the wrong default the moment output has to be maintained, reviewed, or changed by someone other than the person who prompted it.
- Watch for the silent drift: an exploratory prototype quietly being treated as the thing you ship. Name that transition explicitly when it happens, and switch modes rather than letting a discovery artifact harden into a production philosophy by default.
- When work feels too fast, audit the friction you removed. Some pauses — the review before merge, the question about whether the architecture can carry this, the refusal of a correct-but-wrong draft that passes every test — are where judgment was happening. Don't optimize those away just because they slow a velocity dashboard.
- Invest framing effort up front rather than catching problems at review. The cost of a badly framed task now scales with how fast the model executes it, so a wrong frame produces more polished output in the wrong direction before anyone notices. Treat review as the place where standards are actively defended against fast, plausible output — not as a bureaucratic gate.

---

# Chapter 3 — Harnesses, Specs, and Codebases Agents Can Actually Use

When coding agents disappoint, teams usually blame the model.

The model missed a dependency. The model misunderstood the architecture. The model refactored the right function but violated a local convention no outsider could have guessed. The model produced code that technically passed, yet somehow still felt wrong. In the postmortem, intelligence becomes the default suspect.

Sometimes that diagnosis is fair. Models do fail because they are weak, distracted, or simply not yet capable enough for the task. But in production codebases, that explanation is often too flattering to the humans involved. Many agent failures are not evidence that the model is hopeless. They are evidence that the environment was never made legible enough for delegated work.

Ryan Lopopolo puts the inversion bluntly: “The important thing is not the code but the prompt and the guardrails that got you there.” The line sounds almost rude in a field obsessed with generated output. But it captures one of the deepest shifts in AI-native engineering. Once you ask a machine to do implementation work instead of merely suggesting snippets, the surroundings become part of the product. The harness around the model starts determining what kind of work is even possible.

If you want AI to write production software, do not begin by asking what model to use. Begin by asking whether your repository, specs, validations, and workflow are structured well enough for a machine collaborator to operate without constant rescue.

## A small software-factory vignette

Consider a team that starts where many teams now start: with a good model inside an ordinary repo. The repo has no `AGENTS.md`, no checked-in setup script, lint that warns instead of failing, and a test suite a contributor can skip without anyone noticing.

At first the results feel magical. The agent writes tests faster than the humans expect. It handles small UI changes cleanly. It can even land a respectable refactor if a senior engineer hovers nearby and corrects its misunderstandings in real time. So the team expands the scope. They ask it to wire together a new endpoint, touch a migration, update a frontend state machine, and preserve some vague house style that nobody has ever written down.

The watch-for here is the moment scope crosses from tasks one senior can supervise in real time to tasks that run unsupervised. That is exactly where quality stops being merely good or bad and starts being erratic.

The patches arrive with no shared failure signature. One uses a dependency the team would never approve, because nothing in the repo encodes the approved set. Another passes tests but ignores a performance convention learned the hard way six months earlier, because that convention lives in one engineer's memory and not in a lint rule. A third is logically fine yet shaped in a way that makes review irritating and rollback risky. The team says the model is inconsistent. What they really mean is that the workplace is inconsistent — the same task can produce three different answers because the environment never pinned down which one is right.

So they change the workplace instead of swapping the model.

They add explicit setup scripts instead of Slack archaeology. They tighten lint and type gates. They create agent-facing instructions. They check in examples of accepted patterns. They write slimmer task specs before handing work off. They stop relying on “everyone kind of knows how we do migrations here.” The repo becomes less like a haunted archive of past decisions and more like a managed surface for machine labor.

That is the recurring case I will call the software factory, and it shows exactly where the leverage sits. The breakthrough is not that the model suddenly became a genius. It is that the team converted tacit judgment into things the agent can read and run: an approved-dependency list the agent checks, a performance rule promoted from memory to a failing test, a setup script that replaces Slack archaeology. Eno Reyes of Factory AI puts the punchline starkly — when you imagine the fully autonomous flow where a filed bug is picked up, fixed, reviewed, and shipped in an hour, "the limiter is not the capability of the coding agent. The limit is your organization's validation criteria." The model was always good enough for the small wins. What gated the larger ones was how much of the team's own standard had been made mechanical.

## The repo is the real interface

Most coding tools still present themselves through chat. You type a request, maybe select a few files, and wait for the assistant to propose a patch. That interface is useful, but it has a predictable trap: it implies the real problem lives in the prompt, so a disappointing result sends you back to reword the prompt rather than to fix the repo. The tell that you have fallen into it is a prompt that has grown three paragraphs of "and remember to" caveats — every one of those caveats is a standard that should have lived in a file the agent reads on every task, not in the wording of one request.

In practice, the prompt is only the visible tip of a much larger system, and the part you can edit on Monday is the part below the waterline.

That part is the codebase plus everything around it: the setup instructions, the architecture, the naming conventions, the tests, the lint rules, the examples of good patches, the traces of previous reviews, the ADRs, the failure cases, and the rules about performance or security that no single file states explicitly. A human engineer entering a mature repository absorbs these constraints slowly. They ask teammates what matters. They notice which patterns recur. They learn what kinds of changes get approved quickly and which ones trigger suspicion.

An agent gets none of that apprenticeship unless the team encodes it, because the agent has no teammates to ask and no memory of last quarter's incident. Whatever was learned in a hallway is, to the agent, simply not there.

Lopopolo makes the obligation explicit: “Your job is to build systems, software and structures that enable your team to be successful. And to do that, we need to make them legible to those agents that are driving the implementation.” That sentence is more radical than it first appears. It means the team is no longer only maintaining software for other humans. It is also maintaining a working environment for machine contributors. There is a cheap test for legibility you can run today: clone the repo into a fresh container and time how long it takes an agent to reach a green test run from nothing. If the answer is "it can't, because step four lives in someone's shell history," you have found a legibility hole — and every such hole is a place the agent will guess.

That is why documentation, ADRs, examples, and historical breadcrumbs matter so much. Those are not decorative artifacts around the “real” software process. In an AI-native workflow, they become part of the execution environment itself. If human expectations live only in scattered memory, the agent cannot inherit them. If the rules of good work are mostly tacit, the agent will violate them in ways that look mysterious only because the team never externalized its own standards.

This is also why the practical unit of AI coding is no longer the snippet. It is the codebase. Naman Jain of Cursor describes the shift cleanly: “My first project was actually working on generating single line... snippets and my last project was generating an entire codebase.” The practical consequence is a shift in where you spend effort. At snippet scale, the prompt is most of the signal and the repo barely matters. At codebase scale the ratio inverts: the agent makes most of its decisions by reading files you wrote weeks ago, so an hour spent making setup, conventions, and accepted patterns explicit pays off across every future task, while an hour spent perfecting one prompt pays off once.

The core mistake many teams make is to treat code generation as the primary problem and repo legibility as a secondary concern. In reality, the second often dominates the first — so the diagnostic move when an agent disappoints is to invert the blame: before swapping models, ask what a new senior hire would have had to ask a teammate to do this task safely, then check whether that answer exists anywhere the agent can read it. A capable model dropped into a murky repository is like a strong engineer dropped into an organization with no onboarding, inconsistent standards, and no access to prior decisions. You can still get lucky. You cannot count on it.

## Good code contains hundreds of unstated decisions

One reason this problem is easy to underestimate is that most of what separates an acceptable patch from a poor one is non-functional and unstated. Lopopolo gives it a memorable scale: producing a single patch, he says, “probably requires 500 little decisions along the way around the underspecified non-functional requirements that go into producing good code.” The exact number is not the point. The point is that repositories are dense with decisions that matter greatly but are rarely captured in the task description.

Here is the mechanism that turns that density into bad output, in Lopopolo's words: the models “during their training have seen trillions of lines of code that make every possible choice of those non-functional requirements that you could ever imagine.” So when a requirement is left unspecified, the agent does not stall — it samples one of the trillions of conventions it has seen, and there is no reason that sample matches yours. A human fills the same gap by reaching for the local norm they absorbed on the job; the agent has no local norm to reach for.

That is where slop comes from, and it is why “do not produce slop” in a prompt does nothing.

The sloppy patch is rarely the sign of a stupid model. It is the sign of a task whose success criteria were never written down, so the agent picked plausible defaults and you happened not to want them. The fix is not a sharper scolding in the prompt. It is to find the specific decision the agent guessed wrong and promote that one non-functional requirement into a durable artifact — a rule, a test, an example — so the next trajectory inherits it instead of re-rolling the dice.

Once you see the problem this way, the prescription changes. The answer is not only “prompt better.” It is to reduce the amount of silent guesswork that the environment demands. Externalize architecture choices. Store examples of accepted patterns. Make non-functional constraints explicit. Give the system stable ways to discover how this team expects software to be built.

That is what harness engineering means. Not a fancier wrapper around a model, but the systematic conversion of tacit engineering judgment into durable, machine-usable constraints. Lopopolo describes the raw materials concretely: “leaving breadcrumbs, documentation, ADRs, persona oriented documentation around what a good job looks like.” The payoff is that the encoding is done once and reused everywhere — get one engineer to write down what a good QA plan looks like, and, in his words, “every agent trajectory is going to get a good QA plan.” That is the leverage that makes the up-front cost worth it: a tacit standard written down once stops being re-litigated on every task.

The move has a cost you should expect and budget for. Lopopolo is blunt that fixing the environment “requires taking short-term velocity hits” — you stop, find the blocker, put the guardrail in place — before the leverage shows up. Cheap generation, the subject of the previous chapter, is what makes that trade pay: when the act of producing code is no longer the bottleneck, the hour you spend encoding judgment is repaid across every future trajectory that inherits it.

## Specs are not paperwork; they are executable intent

This is where spec-driven development earns its place — not as a documentation preference, but as the artifact that holds the 500 unstated decisions still enough for an agent to read them on every retry.

In a purely human workflow, specs often compete with direct conversation. A strong team can get away with more ambiguity because engineers resolve a surprising amount through meetings, hallway discussions, pull-request comments, and local intuition. In an AI-mediated workflow, that ambiguity becomes more expensive. The failure mode is relying on chat to carry intent: context windows expire, tasks get retried, work gets decomposed into subproblems, and different agents touch different layers of the same system — so any intent that lives only in transient conversation dissolves the moment one of those events fires. The operational test is whether a piece of intent would survive a fresh agent picking up the task cold; if it would not, it belongs in a persistent artifact, not the prompt.

Al Harris of Amazon's Kiro team offers one of the clearest framings in the corpus: “specs are natural language, you're using specs as a control surface to explain what you want the system to do.” Treating the spec as a control surface has a concrete behavioral consequence. When the output is wrong, you do not hand-patch the output; you find the line in the spec that under-specified the result, fix it there, and regenerate — the same discipline you would apply to source rather than to a compiled binary. If you find yourself repeatedly editing generated code that the next run will overwrite, that is the signal the spec, not the patch, is the thing to change.

Harris also describes spec-driven development as “a structured workflow that we push you through to reliably deliver high-quality software,” split into requirements, design, and execution phases. The practical value of the split is that it forces the expensive disagreements early: you settle what problem is being solved and how before any code exists, so the execution phase is not where you discover the requirement was wrong. The spec is not a memo attached to the work. It is the stage gate the work passes through.

A useful spec in an AI-native environment is the place you pre-resolve the decisions the agent would otherwise sample at random. Concretely, that means stating the things a task description usually omits: the problem being solved, the constraints the agent should not have to rediscover by trial and error, and the non-functional requirements that are the first to get dropped — which dependencies are allowed, what the rollback story is, the performance budget, the expected test shape, the compatibility line you cannot cross. A good heuristic for what belongs in the spec is exactly the set of things a reviewer would otherwise flag in a pull request; write those down before generation so they never become a review comment at all.

Seen this way, specs are context compression with an evaluation dividend. They pull intent out of chat history and tribal knowledge into one artifact the workflow returns to on every retry — and because the artifact names a concrete notion of success, you can grade an agent run against it. A vague prompt gives you nothing to grade; a spec gives you the pass/fail line.

This does not mean every ticket needs an elaborate design doc. Over-specification can absolutely collapse exploration into bureaucracy, and small or exploratory work is best discovered through fast iteration — writing a heavy spec there is the wrong choice. The test for when to reach for one: the moment the cost of misunderstanding rises — because the task is large, parallelized across agents, safety-sensitive, or expensive to review — write the spec before generating. Below that line, prompt and iterate; above it, externalize intent first, because that is where a stable representation of intent becomes leverage rather than ceremony.

The counterintuitive part is that a stronger model raises the value of the spec rather than lowering it. A weak generator fails visibly — broken builds, obvious nonsense — and the failure is its own alarm. A strong generator fails plausibly: it produces code that compiles, passes the thin tests you have, and looks right, while quietly making the wrong non-functional choice. The better the model, the more its mistakes hide behind competence, and the more you need an external, written notion of success to catch them. The spec is that notion.

## Agent-ready codebases are designed, not discovered

Eno Reyes is especially useful here because he connects old-fashioned engineering hygiene to an AI-native operating model. He begins with a deliberately basic question: “Do you have some automated validation for the format of your code?... for professional software engineers [it's] like, yeah, of course we do.” Then comes the important turn: “But I think you can go a step further.”

The step further is the part teams miss. Reyes's point is not just that validation should exist, but that the validation surface is now also a tool the agent can extend. You can “ask a coding agent, could you figure out where we're not being opinionated enough about our linters,” and have it generate the missing rule or the missing test. He cites a deliberately low bar from his own team — “a slop test is better than no test” — because once even a rough check exists, the next agent notices it, follows the pattern, and the rules ratchet tighter. So agent-readiness is not a fixed audit you pass once. It is a loop: better agents make the environment more opinionated, which makes the agents more reliable, which frees time to make the environment better still.

That loop only runs if the basics are in place. A practical checklist for the starting state usually includes at least the following:

- a stable folder structure rather than a maze of historical accidents
- explicit setup, build, and run commands that do not rely on oral tradition
- strong type, lint, and test gates the agent can run repeatedly
- architecture decisions stored in files instead of buried in memory
- examples of accepted patterns for tests, APIs, migrations, and reviews
- specs or task briefs stored close enough to the work that they survive handoff
- narrower tools or scripts for common operations where free-form shell access is unnecessary

A concrete first move on the list: most coding agents now read an `AGENTS.md` file at the repo root — Reyes calls it “an open standard that almost every single coding agent supports” — so the cheapest available win is to write down setup commands, the approved-dependency policy, and links to accepted patterns there, where every agent will see them. None of this is glamorous, and that is the point. The largest gains usually come not from frontier prompting technique but from reducing the avoidable ambiguity an agent would otherwise resolve by guessing.

The level of validation also sets a hard ceiling on how ambitious your delegation can get. Reyes is explicit that you cannot safely fan out parallel agents or decompose a large modernization into subtasks until single-task execution succeeds “nearly 100% of the time” — if you cannot automatically tell whether one PR is safe, running twenty in parallel just multiplies the unverified output. So the checklist is not only repo hygiene; it is the gate that decides whether the multi-agent patterns later in this chapter are even available to you.

The same affordances help weaker humans too — new hires, cross-functional contributors, future maintainers — which is why “agent-ready” is less an alien new standard than a sharper test of whether the team encoded its own expectations in reusable form. The agent is exposing the difference between standards the team possesses and standards the team can operationalize, and the gap is measurable: it is every rule an engineer enforces in review that is not yet enforced by a check the agent can run.

## The harness is a workflow, not just a wrapper

It is tempting to imagine the harness as a thin layer around a model: maybe a system prompt, a tool list, a sandbox, and a few guardrails. That is too narrow.

A real harness includes environment setup, repository policy, validation steps, task decomposition, review surfaces, memory of prior work, failure handling, and the sequence in which all those things are applied. Lopopolo's working definition gets at what ties them together: “a good harness is really operationalized around giving the model text at the right time.” The unifying job is timing and selection of context — which file, rule, or example reaches the model at the step where it needs it — which is why a harness is a workflow and not a static config blob. The practical diagnostic is to take one disappointing run and ask, at the step it went wrong, whether the relevant constraint was in front of the model. Usually it was sitting in a file the harness never surfaced at that moment.

The software-factory metaphor sharpens this. Eric Zakariasson of Cursor talks about “building your own software factory,” and the phrase earns its keep by redirecting attention from one-off generation to staged production with checks and roll-up visibility — a factory does not assume every worker can safely improvise in every direction. His subtler, more actionable point is that the factory itself needs a spec: to set it, he says, you would keep “a folder in the codebase” with markdown guidance, best practices, and rules. That is the concrete instruction hiding in the metaphor — the process gets a checked-in home, a directory of rules files, rather than living in human habit. Once it is in the repo, the harness is versioned, reviewable, and inherited by every agent the way code is.

This is also where the book's middle spine starts locking together. Once the harness is the unit of design, two questions follow immediately. How do you know the work it produces is good — which is measurement, the subject of Chapter 4. And how does it hold work together across a task that spans hours and tools — which is runtime and state, taken up later. Chapter 3 does not end the argument; it hands the book directly into Chapter 4.

## Subagents and specialization belong to the harness

The same logic extends beyond a single agent.

One of the most interesting developments in modern coding systems is the move toward specialized roles: research agents, review agents, refactor agents, debugging agents, and broader subagent frameworks that let a larger task be decomposed into parallel, semi-independent work. OpenAI's Codex demo makes the shape concrete. Each subagent is defined by a file giving it a name, a description, a sandbox mode — write-enabled or read-only — and its own instructions. Asked to review a repository of persona files, the orchestrator drops into plan mode, partitions the files into review slices, spins up a fixed pool of reviewers (capped at a concurrency limit — six on the presenter's machine), hands each one its assigned files plus pointers to the relevant repo guidance, then tears them down and collates the results. The pattern is reusable: the same partition-then-collate loop runs a security sweep across a commit just as well as a review across a directory.

The key insight is not merely that parallelism makes things faster. It is that specialization makes process explicit. Defining a review agent forces you to answer, in a file, what it should look at, what tools it gets, and what counts as a finding — questions a generalist prompt lets you leave fuzzy.

So when a team writes a dedicated review agent, a repo-auditing agent, or a migration agent restricted to read-only access plus a single codemod script, it is encoding judgment about how that kind of work should be done, and the role file becomes a durable part of the harness. This mirrors what strong human organizations already do: they do not hand every task to a generalist from a blank slate; they create roles, review structures, and bounded responsibilities so judgment scales past any one person.

But subagents also intensify the need for good scaffolding. More workers without a stronger harness do not create a factory. They create chaos faster. The mistake to watch for: adding parallel agents before you can recompose, inspect, and evaluate their output — at that point you have multiplied generation without multiplying the checks, and the throughput is illusory. The rule of thumb is to add a subagent role only once the recomposition and review surface for its output already exists. And recomposition is not the only missing precondition: the moment subagents do more than read — editing files, running migrations, touching a database — they need *isolated* environments, or they corrupt each other's work before any reviewer sees it. The write-or-read sandbox mode in a role file is the first move toward that; Chapter 6 takes up the full question of per-agent runtime isolation, where the lesson is blunt — for code an agent wrote, a shared environment, and even a plain container, is not a real boundary. That means subagents are not an argument against harness engineering. They are evidence that harness engineering is becoming more important.

## The new advantage is environment design

The marketing of AI coding tools naturally focuses on generation. That is the visible magic. The agent edits a file. It writes a test. It proposes a patch. Those moments are real and often impressive.

But generation is the part competitors can buy. The harness is the part they have to build.

Everyone with a budget can rent the same frontier model; what differs is the environment it runs in. So the advantage accrues to the team that makes the repository legible, promotes non-functional judgment out of senior heads and into rules and tests, treats specs as reusable intent, and invests in validations that let the agent check its own work. Reyes puts a number on the stakes that is worth quoting because it is unusually specific: making this investment, he argues, is where “the real like 5x, 6x, 7x comes from,” and the catch is that “it's a choice that you as an organization have” — the model will not hand it to you.

That is why harness engineering deserves to be a primary discipline rather than a tactical trick. The harness is not a helper bolted onto the codebase; it is being checked into the codebase, versioned and reviewed alongside the code it governs.

The winners in AI coding will not simply be the teams with the strongest models. As Reyes frames it, the limiter is no longer the capability of the agent — it is the organization's own validation criteria, which is to say the workplace the model has to operate in.

That is the bridge into the next chapter. Once the environment can produce delegated work at all, the obvious next question is no longer how to generate more. It is how to know whether the generated work is actually good.

## What to do with this

- When an agent disappoints, invert the blame before swapping models: ask whether your repository, specs, validations, and workflow are structured well enough for a machine collaborator to operate without constant rescue. Treat the unexplained "sloppy patch" as a missing success criterion, not a stupid model.
- Run the agent-readiness checklist on one repo this week: a stable folder structure, explicit setup/build/run commands, strong type/lint/test gates the agent can run repeatedly, architecture decisions stored in files, examples of accepted patterns for tests/APIs/migrations/reviews, and specs stored close enough to the work to survive handoff.
- Stop relying on "everyone kind of knows how we do migrations here." Externalize the tacit standards: check in examples of accepted patterns, write down non-functional constraints, and replace Slack archaeology with setup scripts the agent can read.
- Gate spec-writing by cost of misunderstanding, not by habit. For small or exploratory work, prompt and iterate; once a task is large, parallelized, safety-sensitive, or expensive to review, write the spec first — with constraints and non-functional requirements — so it persists across retries and handoffs.
- Apply the survival test to intent: if a fresh agent picking up the task cold would lose a piece of intent because context expired or work was decomposed, move that intent out of chat and into a persistent artifact.
- Encode the factory's own rules into the repo. Create a folder of markdown guidance, best practices, and rules so process stops living only in human habit, and add a specialized subagent role only once the surface to recompose and review its output already exists.

---

# Chapter 4 — Evals Are the Control System

The obvious failure mode of AI is that it can be wrong. The more dangerous one is that it can look right often enough that the team stops measuring.

A demo works twice. A prototype feels sharp. A coding agent lands a decent patch. A support assistant answers a handful of questions convincingly, and everyone starts speaking in the language of vibes — the system feels promising, maybe even close to ready. That is exactly where the trouble begins, because a promising feeling is not a control loop. Production trust comes from the ability to compare versions, catch regressions, preserve hard-won lessons, and measure whether the system still works when real users, real data, and real edge cases arrive. That is what evals are for, and this chapter argues they are not a side practice you bolt on before launch. They are the operating system of a system you intend to trust.

This is the same argument Chapter 2 made about review, now mechanized. There, judgment under abundance was a human posture; here it becomes an instrument. Once a system does delegated work, you cannot eyeball every output, and the discipline that kept cheap generation honest has to become something you can run.

## The unit of evaluation changed

For most of the short history of AI evaluation, the unit was small: a single completion, a one-line answer, a snippet judged in isolation. That worked while the systems themselves were small. It stops working the moment the system's job grows — and the failure mode is specific. A line-level eval can pass green while the agent's end-to-end task fails, because it scores a fragment the workflow no longer depends on. The way to catch it: take an eval that is passing comfortably and ask whether the thing it grades is actually the deliverable a user receives. If the answer is "no, it grades an intermediate string," the unit is wrong and the green is lying.

Naman Jain, who builds coding evals at Cursor, names the shift in his own work: "coding capabilities have leapt from generating one-line snippets to completing entire codebases with agentic workflows." When the deliverable was a snippet, you could grade the snippet. When the deliverable is a multi-file change across a real repository, grading the diff line by line tells you almost nothing about whether the system did the job. The unit of evaluation has to grow to match the unit of work. A codebase change, a multi-step workflow, a retrieval-heavy research task — each has to be judged at the level it operates, not at the level that happens to be easy to score.

The title insists on *control system* rather than *test suite* because the choice changes what you score. A test suite checks fixed assertions about small units; a control system scores the *trajectory* — the sequence of steps the agent took to reach the answer, not just the answer. Concretely: when grading a multi-file change, do not diff only the final files against a golden patch. Score whether the agent reached the known-good end state, and inspect the path it took to get there, because two runs can land on the same diff while one took a safe route and the other deleted a test to make red turn green. The unit you pick is the only thing you can actually improve against — as Ido Pesok at Vercel's v0 puts it, "improvement without measurement is limited and imprecise."

## Evals are not unit tests — and also are

Pesok titled his v0 talk on the subject with a deliberately blunt claim: "Evals are not unit tests." A unit test encodes a binary fact — the function returns 4, or it is broken. An eval usually encodes a judgment about quality, helpfulness, or fit, so a pass/fail score is a graded opinion, not a correctness proof, and the practical danger is acting on the green as if it were one. Ara Khan, who works on evals at Cline, names the two failure modes that surround this. One camp commits "classic benchmark maxing" — chasing a leaderboard number that, in his words, "won't hold the test of actual real-world evidence." The other camp swings to pure taste and vibes. Both are wrong: "there are right ways to use them, there are wrong ways to use them." The operational takeaway is to track a leaderboard or aggregate score *and* keep a human-labeled slice that catches the cases the aggregate hides — neither alone is a control loop.

Lawrence Jones at incident.io says the opposite — he calls them, flatly, "AI unit tests" — and the disagreement is more useful as a build instruction than as a debate. Jones stores each eval as a YAML file checked in next to the prompt it grades. That single layout decision buys two things at once. For the human, the score stays a graded opinion you read with Pesok's caution, not a green checkmark you trust blindly. For the coding agent, the eval is a file in the repo it can open, copy, and extend, so adding a regression case is a diff, not a trip to a dashboard. The practice to copy: keep evals in version control beside the prompt, one case per file or per row, and make "add a failing case" the same motion as "fix a bug."

## Real tasks beat synthetic cleverness

If evals encode judgment rather than facts, the question becomes where the judgment comes from — and the cheap, wrong answer is to invent fresh test cases in a clean room. Mine them instead. Jain sets the bar: a good eval task "should be natural and sourced from the real world, and then you should be able to reliably grade them." Both halves are operational constraints. *Sourced from the real world* rules out synthetic puzzles you wrote to feel comprehensive. *Reliably gradable* rules out tasks where two reasonable reviewers disagree on pass/fail — if you cannot pin a deterministic check or a clear rubric to a candidate case, it is not yet an eval, it is an anecdote.

The best evaluation sets are rarely written from scratch in a clean room. They are drawn from operational history: the failed support conversations, the difficult research tasks, the painful coding regressions, the edge cases that triggered an escalation. What hurt you in production is far more informative than what looked clever in a benchmark, because it is real, specific, and already known to matter. Jain's team gives a concrete recipe to copy: take a real codebase, crawl its commit history, find the commits that fixed actual problems, and turn each fix into a graded task the agent has to reproduce — revert the fix, hand the agent the broken state, and score whether it gets back to the known-good commit. The escalation logs, incident tickets, and bug-fix commits you already have are an unmined eval set; the work of authoring it has mostly been done for you by the failures themselves. The eval is not a synthetic puzzle. It is a re-run of work that happened.

Samuel Colvin at Pydantic adds the discipline that keeps this honest under the pressure of the GenAI era. "We still want to build reliable, scalable applications," he notes, "and that is still hard — arguably harder with Gen AI than it was before." Human-seeded evals — examples a knowledgeable person labeled because they encode a real failure mode — are unusually valuable precisely because they carry that hard-won knowledge into a form the system can be tested against repeatedly. The seeding is the point. A human who has seen the system fail in a particular way writes the case that catches that failure forever after.

The cost is real: natural tasks are harder to score and harder to maintain than toy benchmarks. But treat that difficulty as a signal, not a deterrent — it is the trap of synthetic benchmarks that they stay cheap to score precisely because they have stopped resembling the work. The decision rule follows: when a benchmark is easy to grade and your eval set is passing comfortably, suspect that you are measuring the convenient unit rather than the real one, and go mine the next painful production failure instead. The more the system does genuine work, the less a synthetic eval can tell you about it.

## Observability and evals are the same problem

Offline evals are only half the loop. The other half is the live system, and Phil Hetzel at Braintrust collapses the two: "Observability and eval, to us, are actually the same problem from a systems perspective." The Raindrop team draws the practical consequence — past a certain complexity you go "from a testing and eval paradigm to a monitoring paradigm," because no fixed offline set can enumerate the edge cases a real agent hits. The thing that makes both possible is the same artifact: the trace. As Arize's Dat Ngo puts it, "code doesn't audit agents or harnesses — it's actually the telemetry that does that." So the concrete move is to instrument every production run as an OpenTelemetry trace from day one, before you think you need it, because a trace you did not capture is an eval case you can never recover.

The usual mental model keeps them apart: evals are the offline thing you run before shipping, observability is the production thing you watch after. Hetzel's claim is that they are one loop. Production traces are not merely debugging artifacts you inspect when something breaks. They are the raw material for tomorrow's regression set. Every real interaction the system has — every success, every failure, every weird edge case a user actually hit — is a candidate eval case, and the strongest teams close that loop deliberately: traces feed failure analysis, failure analysis feeds the eval set, the eval set steers the next version, and the next version is watched in production again. Observability is not downstream of evals. It is where the next generation of evals is born.

This is also why Hetzel insists that "an eval platform is not just a test runner." A test runner executes assertions and reports pass/fail. An eval platform has to hold datasets, persist results across versions, support comparison workflows, render traces beside scores, and produce scoring credible enough to act on. Treat that list as a buy-or-build checklist: if your tool cannot persist results across versions and put a trace next to its score, it is a test runner wearing an eval platform's name, and it will not catch drift. The infrastructure is not incidental to the discipline. It is the discipline, made operable. A team that treats evals as a script they run by hand will measure once, feel reassured, and miss the drift that the loop was supposed to catch.

## When the agents read the evals too

One more design constraint shows up once coding agents start writing and modifying code alongside you: the eval system is no longer read only by humans. If an agent is fixing a bug, the natural next step is for it to also add the regression case that proves the fix — which it can only do if the eval suite is something an agent can open and edit. incident.io is the clearest worked example of building for that. The lesson it teaches is concrete: design the eval suite's interface for the agent that will extend it, not only for the human who reviews it.

Lawrence Jones at incident.io describes building this the hard way. The team stored evals as YAML next to their Go prompt files, and then watched the natural instinct — wrap the evals in richer and richer browser UIs — fail twice over. Humans liked the dashboards but did not have time to use them, and the coding agents could not navigate them at all: "coding agents weren't able to work with them." The unlock was not a better UI. It was a small CLI — "a small CLI tool that we call eval tool, designed to allow agents to leverage our eval suite files." The eval suite became an interface an agent fleet could plug into, rather than a destination a human had to visit.

The same inversion solved their observability problem. incident.io had built rich web UIs to debug AI traces; the agents, again, could not use them. So instead of wrapping the trace database in a fancier front end, they dumped the whole thing as a file tree — because, as Jones puts it, "file systems are exceptionally good agent context." Then they pushed it further: their "scrapbook" pipeline downloads every backtest investigation as a file system and runs roughly twenty-five agents in parallel, one per investigation, clustering the analyses into cohort patterns. The output is not a number on a dashboard. It is a structured improvement report — agents evaluating agent output, with the human receiving a diff instead of a chart. Jones is careful to generalize: "these patterns do generalize" beyond incident response.

This rests on one team's account, so hold it as a worked pattern rather than a law. But the test it implies is portable, and blunt: could a coding agent, given repo access and no human in the loop, find a failing eval, read the trace behind it, and add a regression case — using only the interface you already have? If the honest answer is "only a human with the dashboard open could do that," the eval system is built for the wrong reader. The fix is rarely a better dashboard; it is exposing the same data as files and a CLI an agent can drive.

## Why this is the operating system of trust

Pull the threads together and the chapter resolves into a loop you can build, not a ritual you perform before launch. The loop has four moves: turn fuzzy standards like *good* and *safe* into concrete cases with a rubric or a deterministic check; score the real unit of work and its trajectory, not the convenient fragment; seed the cases from what actually hurt in production, captured as traces; and feed each new production failure back as a regression case so the set grows from the system's own mistakes. Run that loop and the eval set stops being a snapshot and becomes the steering signal — including, increasingly, one a coding agent can read and extend, not only one a human reviews.

The same instrument answers the questions the rest of the book raises. When you change context assembly (Chapter 5), the eval tells you whether the new retrieval actually improved answers or just spent more tokens — A/B the two assemblies against the same task set and read the score, do not eyeball a few outputs. When you run a long, durable task (Chapter 6), the eval that matters is one that resumes the run from a checkpoint and checks the system still behaves after the tenth resume, not just on a cold start. And at the organizational scale (Chapter 9), the eval suite's coverage is what lets a team auto-merge on green instead of routing everything through a human — which is exactly why a thin eval suite quietly becomes the review bottleneck. In each case the eval converts a hope about the system into a number you can act on.

The book's recurring claim is that reliability comes from the scaffolding around the model, not from the model's cleverness. Evals are the part of that scaffolding that tells you whether the rest of it is working. The simplest test of whether you have them yet: can you answer "did the last change make the system better or worse?" with a number rather than a feeling? If the honest answer is a feeling, you are steering blind, and every other discipline in this book is a guess you have decided to believe. With a number you can compare across versions, it becomes something you can measure, steer, and trust.

## What to do with this

- Match the eval's unit to the unit of work. If the deliverable is a multi-file change or a multi-step workflow, stop grading the diff line by line and score the completed task at the level it operates — grading a snippet that no longer exists tells you almost nothing.
- Mine your operational history instead of authoring from scratch. Crawl your commit history for fixes, revert each one, hand the agent the broken state, and score whether it reaches the known-good commit — the way Jain's team builds at Cursor. Your escalation logs, incident tickets, and bug-fix commits are an eval set the failures already wrote for you.
- Seed evals from real failures, not clever puzzles. When someone who knows the system watches it fail in a particular way, capture that case so it catches that failure forever after — that human-seeded knowledge is the part a synthetic benchmark cannot give you.
- Treat a comfortably-passing synthetic benchmark as a warning. When the set is cheap to grade and passing easily, suspect you are measuring the convenient unit, and go mine the next painful production failure instead.
- Close the loop between observability and evals. Treat production traces as the raw material for tomorrow's regression set: route traces into failure analysis, failure analysis into the eval set, the eval set into the next version, then watch that version in production again.
- Audit your tooling against the platform bar. If your eval tool cannot persist results across versions and render a trace beside its score, it is a test runner — it will let you measure once and miss the drift. Also check that an agent, not just a human dashboard, can read and modify the eval suite: a small CLI over a file tree beats a rich browser UI agents cannot navigate.

---

# Chapter 5 — Context Is Infrastructure

Useful AI systems do not fail only because the model is weak. They fail because the system cannot assemble the right working set of information at the right moment, in the right shape, at a cost the product can bear.

For a while, context looked like a prompt-trick problem. You had a box, a token limit, and a growing collection of devices for stuffing more things into it. Add a few retrieved documents. Paste the spec. Prepend some examples. Tell the model to think harder. Each move felt like progress because each one was visible: more characters in, more confidence out.

But the framing was upside down.

Context is not the garnish around intelligence. It is the substrate that determines what the system can even notice. Two systems with identical model weights can differ enormously in usefulness based purely on what reaches the model and what shape that information arrives in. The chapter that follows is about treating that substrate the way engineering treats other substrates: as infrastructure, with versions, budgets, observability, and failure modes you have a plan for.

## Context is the substrate, not the garnish

The earliest framings of prompt engineering trained a generation of builders to think of context as text you assemble in a string, and to assume that if the window is big enough you can keep stuffing. Nupur Sharma's Qodo work shows why that assumption fails on real workloads: models privilege the start and the end of the window and degrade in the middle, so a longer prompt does not buy more attention, it buys a wider blind spot. Her detection cue is concrete — when accuracy drops as you add more retrieved documents rather than rising, you are watching the middle get dropped, and the fix is assembly (hierarchical summarization, graphs, iterative retrieval), not a bigger window. The string-assembly model survives a chatbot answering one question. It breaks the moment the system has to act over real workflows, with real users, against real data, for more than a few turns.

Val Bercovici names the shift directly. Context platform engineering, he calls it, is "the set of skills and tools to design, size, and configure systems optimized for agent swarm context, at any scale." The phrase is dense, but the move is clear. The thing being engineered is no longer the prompt. It is the platform that decides what gets put into prompts. Bercovici's own measure of whether that platform is working is the KV-cache hit rate: when the platform assembles context so that the stable prefix stays identical across calls, the model reuses cached key-value tensors instead of recomputing them, which is where most of the token cost and latency hides. That gives you a usable design rule — order context from most-stable to most-volatile so the cacheable prefix is as long as possible, and treat a falling cache-hit rate as a regression in the context layer, not just a billing line.

Ofer Mendelevitch's enterprise deep-research framing pushes the argument to its limit. The hard problem of enterprise AI, he says, is not access to documents. It is access to the *relevant* documents — the ones the agent actually needs for the current step, ranked, deduplicated, and trustworthy. His own headline number is the reason to take this seriously: in his telling, roughly 73% of enterprise LLM customers name factual accuracy as their top challenge, and the accuracy is lost in assembly long before the model reasons. The operational consequence is a pipeline, not a prompt: retrieve a broad candidate set, rank it, deduplicate near-identical passages, drop stale versions, tag provenance, and only then hand the model the few hundred passages that survive. Once you accept that frame, prompt engineering becomes a small subset of a much larger discipline, and "we have access to the documents" stops counting as having solved retrieval.

## Stuffing context is not memory

Jack Morris has one of the sharpest one-liners in the corpus: "Stuffing context is not memory."

The line is so quotable it can be mistaken for a slogan. It is actually a load-bearing claim about architecture.

A system that only places relevant documents into a prompt has accomplished retrieval, not memory. The model has no commitment to those documents after the response is generated; the retrieval layer itself keeps no durable model of what it surfaced. Unless something outside the window writes state down, the next turn re-derives everything, and the continuity the user feels is manufactured each time by re-stuffing the window with freshly chosen artifacts. This is the operational tell that separates the two: ask whether a fact the user stated three sessions ago can change the system's behavior now *without that fact being re-retrieved into the prompt*. If the answer is no, you have retrieval wearing a memory costume — and the fix is a separate memory store, not a better reranker.

Memory, by contrast, requires durable structure, and the structure is not generic. Chalef's rule at Zep is to model your memory after your business domain: the entities that matter are your customers, accounts, tickets, and the relationships between them, not a flat log of chat turns. Concretely, a memory layer has to decide what to retain versus summarize, what to forget on a schedule, what to refresh when the source changes, and what to surface unprompted — and it has to update an old belief when new evidence arrives rather than appending a contradicting one beside it. Chalef's failure mode is precise: irrelevant facts pollute memory, so a memory store with no forgetting or no belief-update path slowly poisons its own retrievals. None of those properties emerge from pushing more text into a window; they are schema and update-policy decisions you make before the first write.

The reason this distinction matters is that the failures look identical from the outside. A system that retrieves badly and a system that forgets can both surface a wrong answer about a customer the agent talked to yesterday. You can tell them apart with one test: paste the missing fact directly into the prompt and re-run. If the answer comes right, it was a retrieval failure — the fact existed and the pipeline failed to fetch it, so the fix lives in ranking, recall, or freshness. If the answer is still wrong even with the fact in front of the model, or the system never had a place to write that fact down in the first place, it is a memory failure, and no amount of retrieval tuning will patch it. Better retrieval fixes the first; only a memory architecture fixes the second.

Daniel Chalef at Zep makes a related point with a more pointed framing: stop using RAG as memory. The error he sees in production systems is not RAG itself but RAG carrying weight it was never designed to carry — long-term user state, evolving entity facts, cross-session continuity. RAG is good at fetching documents. It is bad at maintaining a model of the user across months.

Read these two claims together as a selection rule. Reach for retrieval when the job is fetching the right documents for the current step; reach for a memory layer when the job is maintaining state that has to persist — long-term user facts, evolving entities, cross-session continuity. Chalef's warning marks the trap: the moment RAG starts carrying that durable state, you have collapsed two layers that need to be designed separately.

## RAG, memory, and GraphRAG are different jobs

Once the distinction between retrieval and memory is on the table, the next obvious question is whether all retrieval is the same. It is not.

Stephen Chin's GraphRAG work and the broader Neo4j framing argue that retrieving over a graph is a different operation than retrieving over a flat vector index. The vector index excels when the question is about semantic similarity to existing content. The graph excels when the question is about relationships — who depends on whom, which entities co-occur, what does this concept connect to. Mitesh Patel at NVIDIA pushes the framing further with hybrid RAG, which fuses graph and vector retrieval because most useful enterprise questions need both.

David Karam, from his Pi Labs work after Google Search, frames retrieval as a layered problem: you don't pick one technique, you layer them one query at a time, and each layer must be tuned to what it gives you before you invest in it. The layers are specific and have specific failure classes. Vector search catches semantic paraphrase but misses exact identifiers and rare tokens. BM25 or full-text catches the exact clause number or error string that embeddings blur away. Grep, regex, and metadata filters catch the cases where the right answer is a literal match the ranker would have buried. A reranker reorders the merged candidates so the most relevant rise before the window fills. Kuba Rogut's blunt version at Turbopuffer is that retrieval is not just vector search — and the practical consequence is that the engineering job is mapping each known failure class to the layer that catches it, then measuring recall per layer rather than trusting the stack as a black box.

The deeper claim is that the field's vocabulary has been lagging the architecture. Practitioners say "RAG" and mean four different things depending on context. Sometimes they mean a single embedding lookup before a single prompt. Sometimes they mean a retrieve-then-rerank pipeline. Sometimes they mean a graph traversal that surfaces entities. Sometimes they mean a long-term memory layer. The label has collapsed distinctions that the architecture depends on.

Will Bryk's neural-RAG work at Exa comes at it from the opposite direction. His bet is that "the right agent in the future is going to be this system that decides what type of search" — retrieval folded into the reasoning loop instead of run once before the prompt. The model issues a query, reads the result, decides whether to follow up, and decides when to stop. That is also called RAG today, but it shares almost nothing with a single embedding lookup, and the trade-off is concrete: iterative retrieval buys recall on open-ended research questions at the cost of many more model calls and far higher latency. So the decision rule is by query shape, not by fashion — use one-shot retrieve-then-rank when the question is bounded and you can name the right index up front; reach for loop-integrated retrieval only when the agent genuinely cannot know which searches it needs until it has read the early results.

The practical consequence is that "we'll just add RAG" is no longer a useful sentence in a design discussion, because the word now hides at least four different builds: single embedding lookup, retrieve-then-rerank, graph traversal over entities, and a long-term memory layer. Make the design discussion answer five questions explicitly before anyone writes code — which of those patterns, against which data shape (flat documents, relational records, or a graph), with what ranking and dedup, with what freshness guarantee on stale versions, feeding into which reasoning surface (one-shot or loop-integrated). A design doc that cannot fill in those five blanks has not specified a retrieval system; it has named a buzzword and is relying on luck.

## Enterprise usefulness is a working-set problem

The argument so far has been mostly about technique. The harder argument is about value.

Joel Hron's framing — that AI is shifting from helpfulness to producing judgments — looks different inside an enterprise than it does inside a consumer chatbot. Inside the enterprise, the agent's value depends almost entirely on its ability to assemble a small, accurate, current, trustworthy working set out of a much larger, messier corpus. Without that working set, the model is doing impressive cognition over the wrong material, and the answer is wrong in a way that is hard to catch.

Kuba Rogut puts the sizing rule in one line, relaying Jeff Dean: you don't need a trillion tokens at once, you need the right million. That reframes the budget question. The number to instrument is not how many documents the index holds but how much of the assembled working set the model actually used to produce a correct answer — and when a broad query drags in hundreds of passages, the work that produces value is the convergence on the handful that matter, not the cognition that happens after. If you cannot say which retrieved passages the answer depended on, you cannot tell a convergence win from a lucky guess.

Calvin Qi at Harvey and Chang She at Lance describe the same pattern from inside legal work. Lawyers do not need an agent that can read everything. They need an agent that can find the specific clause, the specific precedent, the specific exception that bears on the matter at hand. The retrieval has to separate authoritative sources from background material, and it has to surface provenance so the lawyer can verify what the system found before relying on it.

Chau Tran's Glean work generalizes this across enterprises. The enterprise-aware agent, in his framing, is not one that has access to the company's documents. It is one that knows which documents matter for the current user, the current role, the current task — and which to ignore. The boundary work is the engineering work, and it is mostly filtering on signals the corpus already carries: the user's permission scope (so the working set never includes documents this person cannot see), the freshness of the source, and the document's role in the org rather than its raw text-similarity score. The trap to avoid is ranking purely on embedding similarity, which happily surfaces a deprecated wiki page that reads almost identically to the current one. Permission-scope and recency filters belong before the reranker, not after, so excluded material never competes for the window in the first place.

The unifying claim across these talks is that enterprise usefulness scales with working-set quality, not with corpus size. A larger corpus without better assembly produces worse outcomes, not better ones. A smaller corpus that is well-ranked, well-scoped, and provenance-tagged often outperforms a larger one that is dumped in raw.

The working set is not only a quality lever; it is the dominant *cost* lever — the version of this argument with a dollar figure attached. Rajkumar Sakthivel's team at Tesco states the decomposition bluntly: "90% of your AI cost is input. Files, search results, context you send in. Only 10% is output." That inverts the usual optimization instinct, which reaches first for a cheaper model: "the model choice matters less than you think," he argues, because the model "may be 30% of the cost, but other 70% is what you feed it." Indexing a codebase and retrieving the relevant slices instead of pasting whole files cut their input tokens by a measured — if best-case — 94% on a benchmark repo. Stuffing the window is not just worse cognition; it is the line item.

This is also where Chapter 4's argument should still be echoing, and it implies a specific eval design. Evals can measure whether an answer is right, but context architecture determines whether the right answer was even reachable. So split the metric: score retrieval and generation separately. Track whether the gold passage made it into the assembled working set at all (a recall measure on the context layer) before you score whether the model used it correctly. When end-to-end accuracy drops, that split tells you which half regressed — a model you can swap, or a substrate you have to fix. Score only the final answer and a context-assembly bug looks exactly like a model getting dumber, and you will waste a model upgrade on it.

## The next failure frontier is context misassembly

For most of the public conversation about AI quality, hallucination has been the boogeyman. The model invented a citation. The model fabricated a fact. The model imagined a precedent that does not exist. Hallucination is real and worth measuring — and a grounding check that verifies every claim traces to a cited source will catch most of it. But that exact check is also why hallucination is becoming the wrong primary failure mode to obsess over: the frontier failures pass it. Every source is real, every quote verifies, and the answer is still wrong. A citation-grounding eval scores those cases as clean, which is precisely how they slip into production.

The newer, more expensive failure mode is context misassembly.

Context misassembly is what happens when the system retrieves real documents, in the wrong combination, with the wrong weighting, at the wrong moment, and produces an answer that is technically grounded but practically misleading. Nothing is hallucinated. Every cited source exists. Every quote can be verified. But the assembled context misrepresents the underlying state of the world because the assembly missed something, ranked something poorly, or surfaced an outdated version of a document the system also has the current version of.

Morris's distinction between stuffing and memory points at one form of this. The system surfaces three documents about the customer, two of them stale, one of them current. The model averages them and produces an answer that is half-current. Nothing is invented; nothing is correct either. The fix is not a smarter model but a dedup-and-recency pass before assembly: when two retrieved chunks describe the same entity, keep the most recent and drop the rest rather than letting both into the window to be averaged. The detection cue is a near-duplicate alarm — if your retrieval routinely returns multiple high-similarity chunks that differ mainly by date, you are feeding the model contradictions and calling it context.

Ivan Leo's Manus AI research-agent work — now under Meta Superintelligence — surfaces the same problem at a different scale. A deep-research agent that pulls hundreds of sources can produce a summary that is internally consistent and externally wrong because the assembly drifted as the agent worked. Each individual retrieval was fine. The composition was off.

Karam's layered-RAG approach is partly a response to this. By treating retrieval as a multi-pass operation with distinct failure modes per layer, the system gives misassembly multiple chances to be caught by a downstream layer. That works, partially, but it does not change the underlying architectural fact: context misassembly is a structural failure mode, and the systems that are starting to dominate production are the ones that designed for it explicitly.

The reason this matters for the book's overall argument is that misassembly does not get fixed by a better model. A better model produces a more confident wrong answer faster. The fix lives in the substrate — in how the system assembles, ranks, deduplicates, and freshens the context before the model is asked to reason over it. The checks that catch it run on the assembled working set, not the output: was a stale version included when a current one existed, were two retrieved sources contradictory, did the gold passage rank below the window cutoff. Those are misassembly tells an output-only score waves straight through. That is what makes context engineering an infrastructure problem instead of a prompt problem.

## MCP makes context a capability problem too

The rise of the Model Context Protocol has expanded what context includes. An MCP-connected agent has access not only to documents but to tools — APIs, search endpoints, file operations, internal services, and increasingly, other agents. Each of those tools shows up in the context window as a capability description: a name, a schema, an example. The window now contains both the information the system might consult and the actions the system might take.

That expansion brings a new failure mode.

Matt Carey's MCP mega-context-problem talk identifies the failure cleanly: "We shouldn't be dumping loads of tools into context." When an agent has access to fifty tools, the capability descriptions for those tools can flood the window before the user's question is even processed. The model spends its attention budget reading tool definitions and has less budget for the actual reasoning the user wanted. Worse, the abundance of tools tempts the model into calling the wrong one because it now has to disambiguate between options that all look superficially relevant.

Sam Morrow's GitHub MCP-scaling work tells the production-grade version of this story, with numbers. Community contributions pushed GitHub's server past 100 tools and the agents got measurably worse — windows blew out, models got confused and forgetful — the same failure LangChain had already published. GitHub's first instinct was elegant opt-in machinery: tool sets, dynamic discovery, even a RAG-based tool search. Almost no one used any of it, because most users never touch the JSON config. That is the load-bearing lesson: any fix that depends on user configuration reaches a minority, so change the default instead. GitHub did, cutting the initial tool-load context by 49% by analyzing real usage rather than guessing, then trimming output too (list-pull-requests shed 75% of its output tokens by dropping unused fields). The number of tools the agent could in principle call did not go down. The number it had to read, and the tokens each one cost, did. That difference is the engineering.

Karan Sampath, sharing what Anthropic learned scaling MCP to enterprises, points at the same dynamic from the governance side. The enterprise version is not just performance, it is trust, and his prescription is concrete: a security team's goal is to "bless one platform" and "establish a root of trust" rather than vet servers one at a time. The architecture that follows is recognizable — a gateway, a registry of reviewed servers, tools scoped to the calling principal, and an audit log at the gateway layer. The point for context design is that this is the same shaping problem seen from a different seat: deciding which tools a given principal sees is both a capability-flood decision and a permission decision, so the filter that trims the window for performance is also the surface the security team inspects. Build them as one layer, not two.

The unifying claim here is that the moment tools entered the context window, context engineering became broader than retrieval; it also became capability management. The directive the production cases point to: expose tools by intent rather than all at once, describe them tightly, and retract them when they stop being relevant — so you shrink the number of tools the agent must read at any moment, not the number it can call.

## Progressive discovery is infrastructure, not UX

The natural reaction to the tool-flood problem is to surface fewer things up front and let the agent discover more as it needs them. The technique has a name — progressive disclosure — and Pedro Rodrigues at Supabase states the mechanism plainly: a capability "doesn't have to be loaded immediately to context." You expose a short index of what exists, and the full schema, examples, and instructions for a tool or skill load only when the agent reaches for it. His tested result is the reason to bother: MCP paired with skills using progressive disclosure beat raw MCP alone, because raw MCP pays the full token cost of every tool up front. The harder claim is that this deserves to be treated as infrastructure rather than a UX nicety.

Bercovici's context-platform framing makes this argument structurally. The right amount of context to expose at any given step is a function of the step, the agent, the user, and the task — not a static property of the system. That is exactly the shape of an infrastructure problem. It needs a layer that owns the decision, observes the outcome, and updates over time.

Sam Morrow's GitHub work is the closest thing the corpus has to a production case study for this. The team did not just trim the tool list. They built grouping, tailoring, and intent-aware exposure into the MCP server itself. The agent does not see every tool; it sees the tools the server has decided are relevant given the agent's intent. The decision lives in the server, not the prompt.

The deeper point is that progressive discovery does work the model cannot do for itself. The model can reason about the tools it is shown; it cannot reason about tools absent from the window. So the selector — whatever decides which tools to show — is part of the architecture, and it has to be testable like one. That means logging, per request, which tools were exposed and which the agent actually called, then watching two failure signals: tools that get exposed every time and never called (dead weight you can drop from the default) and tasks that fail because the needed tool was never surfaced (a recall miss in the selector). Both are measurable, and neither is something the model can fix from inside the prompt.

This is also where the book's earlier claims about harness engineering — Chapter 3 — start cross-loading. "Agent-ready codebase" stops meaning "a repository with good tests" and starts including the capability surface: how many tools the agent sees by default, whether their schemas are tight enough that the agent picks the right one, whether stale docs are excluded from retrieval, whether the AGENTS.md and skill files load progressively instead of all at once. A concrete first audit is to count the tools and context your harness loads on an empty task — if the agent reads fifty tool definitions before it has a goal, you have a context-platform problem wearing a harness label, and the fix lives in the server and the retrieval layer, not in the prompt.

## Why context is infrastructure

Context is the substrate, not the garnish.

If context is a substrate, then context engineering is the discipline of building, maintaining, and observing it — and each of its mechanisms fires on a recognizable trigger. Retrieval architecture answers "fetch the right documents for this step." Memory architecture answers "carry durable user and entity state across sessions." Capability management and progressive disclosure answer "expose only the tools this step needs." Freshness handling and provenance answer "make sure the version is current and its source is traceable." Cost and latency budgets — KV-cache hit rate, tokens per call, layers per query — decide when each of the others is allowed to fire. It is not a thing one component does. It is a layer with its own state and its own failure modes, and naming the trigger for each mechanism is how you stop treating them as one undifferentiated "RAG" knob.

A team that treats context as a one-off prompt-assembly problem will keep finding the same failures and keep fixing them with one-off measures. Add a reranker here. Patch a deduplication bug there. Re-tune a chunk size when retrieval starts missing the relevant clause. None of those are wrong, but none of them add up to an architecture.

A team that treats context as infrastructure builds for the failure modes the chapter has named: misassembly, capability flood, memory drift, provenance loss. They version the context layer. They observe what it surfaces. They measure how the model's outputs change when the context layer changes. They treat the context platform the way a database team treats the storage engine — as a piece of working infrastructure whose properties shape everything built on top of it.

Treating context as infrastructure has a structural consequence the next chapter takes up: the substrate cannot live inside a single agent run. The moment memory has to survive across sessions and a half-finished task has to resume after an interruption, the context layer needs somewhere durable to persist and a way to replay — which is the runtime and state problem, not a context problem. That threshold, where a stateless prompt becomes a stateful workflow, is where Chapter 6 begins.

## What to do with this

- Treat context as a versioned, observable layer with its own budgets and failure modes, not a string you assemble per request.
- Don't use RAG as your memory layer: retrieve to fetch the right documents for a step, and design a separate memory architecture for durable user and entity state across sessions.
- Match the retrieval pattern to the question — vector search for semantic similarity, graph traversal for relationship questions, hybrid when the query needs both — and make ranking, data shape, and freshness explicit before you build.
- Optimize for working-set quality, not corpus size: a smaller, well-ranked, provenance-tagged set beats a larger raw dump.
- Budget cost at the input, not the model. In a retrieval or coding agent, the prompt you assemble — files, search results, history — usually dominates the bill, not the tokens the model writes back. Index and retrieve precise slices instead of pasting whole files, cache stable prefixes, and summarize tool results before re-sending; fix the input before you reach for a cheaper model.
- Design for context misassembly, not just hallucination — dedupe, freshen, and weight retrieved documents so stale-and-current sources can't average into a half-right answer. A stronger model will not fix this.
- Don't flood the window with tools: expose them by intent and retract them when irrelevant, shrinking the number of tools the agent must read at any moment rather than the number it can call.

---

# Chapter 6 — Runtimes, State, and the Human Control Plane

A chatbot can get away with amnesia. A production agent cannot.

Short-lived assistance can live inside a conversational loop — a turn comes in, a turn goes out, and nothing has to survive past the reply. Delegated work cannot live there. The moment a system has to persist across retries, timeouts, approvals, multiple tools, and possibly many parallel workers, the central problem stops being next-token cleverness and becomes execution semantics. The system has to preserve state, survive interruption, expose its progress, and resume without losing the thread. This is the chapter where the book's argument turns from what the model knows to how the work itself stays alive between the moments the model is thinking.

Samuel Colvin at Pydantic states the lesson in the voice of someone who learned it in production: "building production AI agents reveals a harsh truth — stateless architectures that work for simple demos become impossibly painful at scale." The word *painful* is doing honest work. The failure is not dramatic. It is the slow accumulation of edge cases — the timeout that loses an hour of work, the retry that double-charges, the approval that arrives after the agent already acted — that turns a magical demo into a system nobody trusts.

## Why stateless demos break

The seductive thing about a chat-loop agent is that it works immediately. You wire a model to some tools, let it loop, and on a short, happy-path task it looks like the future. The trouble is that the architecture has no memory of what has actually happened — only a transcript of what was said. And a transcript is not state.

The distinction matters the instant anything goes wrong. A real delegated task runs long enough to hit a timeout, a rate limit, a flaky tool, a network blip, a required human approval. In a stateless loop, every one of those is a small catastrophe, because there is no durable record of where the work had gotten to. The system cannot answer the only questions that matter under failure: what has been done, what is in flight, and what is safe to retry. It can replay the conversation, but it cannot resume the work, because it never modeled the work as something separate from the conversation about it.

This is why so many impressive agent demos collapse when a team tries to operationalize them. The model is capable, the prompts are decent, the context is strong — and the system still falls over, because it was built like a chat session when the job required a workflow. So run the test before you scale a demo: ask whether the system can answer what has been done, what is in flight, and what is safe to retry without replaying the conversation. If the only record is a transcript, you have a chat session, not a workflow — and the fix is not a smarter model but a different architecture underneath.

## Durable execution is the runtime requirement

The architecture that survives reality is durable execution: treat the long-running task as structured, checkpointed execution rather than a growing transcript. Preeti Somal at Temporal puts the stakes plainly: agentic systems "must scale and provide durability and reliability — otherwise, no one's going to trust your agent." Trust, in her framing, is not a property of the model's answers. It is a property of the runtime's behavior under failure.

What durability buys is the ability to distinguish what was said from what has actually happened. A durable runtime records each completed step, so that when a tool call times out on the ninth step of a twelve-step task, the system resumes from the ninth step rather than restarting from zero or, worse, redoing a side effect that already fired. Somal describes pushing the reliability semantics down into the runtime so they leave the prompt entirely: "nowhere in there will there be statements like, if something fails keep retrying it — all of those pieces are handled" by the execution layer. That is the architectural move. Retries, timeouts, and resumption stop being things the agent has to reason about turn by turn and become guarantees the runtime provides. The prompt gets to be about the task; the runtime takes care of survival.

The same talk names the second thing durability gives you, and it points straight at the next section: "we also store all of the workflow history, so that you can look at the visibility of what is happening as your agent is navigating this complex set of interactions." History is not just for recovery. It is for inspection — and inspection is where humans re-enter the system.

## The human control plane is an architectural layer

It is tempting to treat human oversight as a temporary crutch — something you keep around until the models get good enough to remove it. The corpus argues the opposite. In high-value systems, human control is not a phase on the way to full autonomy. It is a permanent architectural layer, designed in rather than bolted on.

Joel Hron at Thomson Reuters gives the clearest frame for what that layer regulates. Autonomy, he argues, is not a switch you flip but a dial you tune: agency is "not a binary thing but a lever that you can dial" up or down depending on how irreversible, risky, and observable the work is. A low-stakes draft can run with the dial turned far up. A filing with professional consequences runs with it turned down, with explicit approval points where a human re-enters before the system acts. The design question is never "autonomous or not." It is "how much agency, at which steps, with what review."

The mechanism that makes those approval points workable is exactly the workflow history durability provides. Hron describes deep-research systems whose long-running behavior becomes "the trajectories that the model would be following along its path of answering this particular type of legal question" — inspectable paths a human can audit rather than opaque jumps from question to answer. The control plane is built from these surfaces: the approval gates where a human authorizes the next step, the roll-up views that show what the system is doing without drowning the reviewer in raw agent chatter, the trajectory and history records that make an action reconstructable after the fact. Eric Zakariasson at Cursor describes the roll-up form of this directly — the human needs "an overview of the processes," a single surface answering what every worker is doing and what actually needs a person's attention, rather than a firehose of individual logs.

This is the same human-judgment-at-the-edges principle the book keeps returning to, now given a concrete home in the runtime. The point of the control plane is not to slow the system down. It is to focus scarce human attention on the consequential moments and let the runtime carry everything else — which gives you a design test for any review surface: if it surfaces raw agent chatter instead of a roll-up of what needs a person, or if it gates a reversible low-stakes step the way it gates an irreversible one, it is spending human attention in the wrong place.

Attention is not the only scarce resource the control plane rations. Compute is the other, and the same logic applies: match the cost of the response to the difficulty of the request rather than paying frontier prices for every step. Practitioners treat tiered model selection as settled rather than novel. Laurie Voss at Arize states it almost prescriptively — use "cheap models for simple queries and expensive models in your agent ... for complex queries" — and Harrison Chase at LangChain describes a router whose job is to "route between ... language models," picking one per question by its strengths, long context against reasoning. The platforms expose the same trade as service tiers: Guillaume Vernade at Google DeepMind describes a flex tier that gives "a 50% discount but your request can be ... delayed," for work that is not latency-sensitive. The decision of which model runs a given step is a control-plane decision, not a global default chosen once. The catch is that routing adds its own failure surface — a misroute hands a hard task to a cheap model that quietly botches it — so aggressive routing is only safe behind the verification the control plane already runs. You can route down to the cheapest model that still passes the eval, and no cheaper.

## Parallelism raises the stakes on coordination

Everything so far concerns a single durable worker. The architecture gets harder, and more interesting, the moment teams reach for many workers at once — and this is where the corpus is most unsettled.

The appeal is obvious: if one agent is leverage, a fleet is more leverage. OpenAI's Codex team describes the mechanism, spinning off "a master task into decomposable parallel and independent tasks." But the precondition is in that phrase: parallelize only work that is genuinely *decomposable and independent*, and only after you have a way to recompose, inspect, and route the output back to a human at the right moment. Add workers before you have that, and parallelism just manufactures chaos faster — more diffs, more conflicts, more review than anyone can absorb, which is exactly the alignment-debt failure Chapter 9 will name at organizational scale. Teams go wrong treating "spin up more agents" as the win when the recomposition layer is the actual bottleneck.

Independence also has to be enforced by the *environment*, not just declared in the task split. Several agents against one shared dev setup collide on the same branch, ports, and database — one agent's migration breaks the others mid-run — so each needs its own isolated, ephemeral environment. Maggie Appleton at GitHub gives each session one "backed by a micro VM ... a sandboxed computer in the cloud on its own Git branch," which is precisely what lets a developer "work on parallel tasks and instantly switch between them." And teams running agent-written code are blunt that the obvious primitive is the wrong one: Rene Brandel at Casco warns that "if you just use containers ... that's not an isolation layer," because agent code can get root and move laterally. Worktrees suffice for trusted edits; untrusted, side-effecting agent work wants a VM.

Lou Bichard at Ona sharpens the diagnosis to a single missing piece. The runtime, he argues, is solved — "there are many options for this now, sandboxes and containers" — and so are triggers and orchestration. "The thing that's missing," he says, "is coordination": the agent-native primitive that lets parallel workers pick up tasks, signal completion, and hand off without a human stitching them together. And he is pointed about what is *not* that primitive: "GitHub is not a coordination layer for agents — it gets incredibly overwhelming." His candidate building blocks are exactly this chapter's subject — "state machines, by building out workflows and effectively state machines" — which lands the parallelism question squarely back on durable execution.

What makes the corpus credible here is that the teams shipping production multi-agent systems have not agreed on an answer; they have each substituted a known mechanism for the missing one. Factory runs features serially with one active writer — "serial execution with targeted internal parallelization" — eliminating the coordination problem by construction, and reports a longest mission of sixteen days. Anthropic's long-running agents take a third path: a planner-generator-evaluator loop where each role gets "its own kind of context window" and the agents "negotiate what done actually means" through a contract written to files on disk before any code is produced — a capability curve their team traces from roughly a one-hour autonomous run to twelve hours on the same simple harness. Serial execution, file-based contracts, state machines plus durable execution: three substitutes for one primitive that does not yet exist. The honest chapter names the gap and shows the three things teams actually ship, rather than pretending there is a consensus building block.

## The runtime is where intelligence becomes dependable

Pull the chapter together and its claim is structural. A machine colleague is not a model with tools attached. It is a model inside an operating environment — and that environment is what determines whether bursts of intelligence become dependable delegated work.

The environment has named parts now. Durable execution, so the work survives interruption and resumes instead of restarting. Explicit state and workflow history, so the system can answer what has actually happened. A human control plane of approvals, roll-up views, and inspectable trajectories, so people supervise at the consequential edges. And, once there are many workers, a coordination story — even if today that story is a chosen substitute rather than a solved primitive. None of these live in the model. All of them live in the runtime, and all of them are the difference between a demo and a system.

The moment a durable, long-running agent can act on its own — with state, tools, and the authority to use them over time — bounding that authority becomes the price of letting it act at all. Identity, permissions, sandboxes, audit: that is the next chapter's subject. Durability gave the agent staying power. Security decides what it is allowed to do with it.

## What to do with this

- Before you scale any agent demo, run the failure-state test: can the system answer what has been done, what is in flight, and what is safe to retry — without replaying the conversation? If the only record is a transcript, you have a chat session, not a workflow. The fix is a different architecture, not a smarter model.
- Push retry, timeout, and resumption semantics down into the runtime so they leave the prompt entirely — aim for a state where "if something fails keep retrying it" appears nowhere in your prompt because the execution layer handles it. Checkpoint each completed step so a tool timeout on step nine resumes from step nine, not from zero and not by re-firing a side effect that already ran.
- Store full workflow history, and use it for two distinct jobs: recovery (resuming after failure) and inspection (a human auditing what the agent actually did). Treat the trajectory record as the surface humans review, not an afterthought.
- Stop asking "autonomous or not" and instead set the agency dial per step — tuned by how irreversible, risky, and observable that step is. Turn it up for a low-stakes draft; turn it down with an explicit approval gate before an action with real consequences.
- Audit your review surfaces: if one shows raw agent chatter instead of a roll-up of what needs a person, or gates a reversible step as strictly as an irreversible one, it is spending scarce human attention in the wrong place. Build the roll-up overview that answers what every worker is doing and what actually needs review.
- Don't reach for more parallel workers until you have a recomposition layer — coordination is the missing primitive, and GitHub is not it. Until an agent-native coordinator exists, pick a deliberate substitute: serial execution with one active writer, file-based "negotiate what done means" contracts, or state machines plus durable execution.
- Before you run agents in parallel, give each one its own isolated environment, not a shared one. Pick the isolation rung by trust and blast radius: a git worktree for trusted edits, a micro-VM for untrusted or side-effecting agent code. A container is not an isolation boundary for code an agent wrote — if it can get root or touch a shared database, it is not contained.
- Treat model choice as a per-step routing decision, not a global default. Route each step to the cheapest model that still passes its eval, escalate only on failure, and use platform flex/batch tiers for latency-tolerant work — but only once verification can catch a misroute, or you have traded cost for silent errors.

---

# Chapter 7 — Security, Identity, and High-Stakes Trust

Once AI systems can act, trust stops being only a question of model quality. It becomes a question of bounded authority.

A helpful model can get away with being vague about power. An acting system cannot.

The moment an AI system can call tools, execute code, traverse accounts, or continue working after the user has stopped watching, security moves to the center of the product. The important question is no longer whether the model sounds smart. It is whether the system has a clear identity, narrow permissions, real execution boundaries, and enough evidence left behind that a human institution can understand what happened later.

That is why agent security is not mainly a prompt-safety problem. It is a delegated-authority problem. Traditional software security often focused on discrete requests. Agentic systems stretch the unit of risk across a workflow: interpretation, retrieval, tool use, retries, follow-up behavior, and approval boundaries. Trust has to cover the whole path.

The strongest support for this claim in the current corpus is not a single definitive study. It is a pattern of practitioner convergence. Security talks, identity talks, and high-stakes workflow talks keep arriving at the same practical lesson from different directions: once systems can act on behalf of users, the hard part is no longer only generating a plausible next step. It is controlling what powers were delegated, under what conditions, and with what record of use.

## Identity is the substrate of trust

The most common engineering shortcut for connecting an agent to a real system is to give the agent a standing credential — a long-lived API key, a service-account token, a personal access token "borrowed" from the human operator. The shortcut works, in the sense that the system can act. It also dissolves the question the chapter is trying to take seriously.

A standing credential is not a delegation. It is the agent inheriting the entire authority of whoever's credential it borrowed, for as long as the credential remains valid, across whatever surface that credential touches. There is no scope to revoke. There is no expiry that maps to the actual task. There is no record that an agent was acting, rather than a human acting on the same key. The unit of trust the system has just granted is much larger than the unit of work it was asked to do.

Patrick Riley and Carlos Galan at Auth0 frame this directly. Their work on identity for AI agents starts from the position that "we authorize agents, MCP servers" — making the agent and its capabilities first-class citizens in the identity provider, not invisible passengers on a human's credential. The shift looks technical but it is structural. Once the agent is a real principal in the identity layer, it can have its own scopes, its own lifetimes, its own audit footprint, and its own revocation path.

Jared Hanson at Keycard pushes the same argument from the developer side. The common habit of treating an agent like an API consumer — with a permanent key paired to a permanent identity — does not survive contact with workflow reality. Agents act on behalf of specific users, for specific tasks, with specific data. Hanson's case is that securing agents using OAuth is not just a protocol choice; it is the right shape for the delegation. The agent gets a token that is bounded to a session, a user, and a set of scopes. When the work is done, the authority ends.

Garrett Galow at WorkOS pushes the framing further. His talk on cross-app access for MCP proposes the identity provider as a *trust bridge* between MCP clients and servers — so that the credentials flowing through the agent can be obtained without repeated manual consent flows, and so that the IT organization keeps visibility into which third-party systems are reading which data. That is the structural answer to a question every team running agents at scale eventually faces: how do you let an agent move between tools without making each tool feel like an independent integration project?

The shared move across these three talks is to treat agent identity as a first-class engineering object — not a label, not a service-account workaround, but a principal in the identity system with bounded scope, bounded lifetime, and an audit footprint. An agent that is authenticated as a blurry extension of a human is not delegated. It is impersonating.

## A concrete failure case

Imagine a tax-preparation agent operating inside a professional workflow of the kind described by Joel Hron at Thomson Reuters. The system ingests source documents, extracts fields, maps them into a tax engine, checks validation errors, revisits documents to resolve missing information, and assembles a draft return. Nothing in that flow requires the model to be malicious for trouble to begin. It only has to be slightly wrong in a place where authority and evidence are easy to blur.

Suppose one document is ambiguous, a number is mapped to the wrong field, and the validation engine throws an error. The agent now has several possible powers: search for more supporting material, pull additional client records, reinterpret the rule, override a warning, or proceed with a draft that looks internally coherent. In a weak design, those powers can collapse into one another. The same system that is supposed to summarize evidence also has enough access to fetch more data, make a judgment call, and keep moving. From the outside, the workflow still looks smooth. The danger is not theatrical failure. The danger is an authority boundary disappearing inside a competent-looking trajectory.

In high-stakes work, the risky move is often not one bad answer. It is a system quietly crossing from assistance into authorization.

## Least privilege is different for agents than for users

Least privilege is the oldest piece of security advice, and on its own it is not enough.

In classic SaaS software, a narrow scope usually constrains a bounded API action. In agentic systems, the same scope can be recombined across many steps. A permission that seems harmless in isolation can become more powerful when paired with retrieval, reasoning, persistence, and retries. So "only give the agent the minimum access it needs" is correct but incomplete. Teams also have to ask: minimum access for which stage of the workflow?

This is where Hanson's argument compounds with Galow's. A standing credential gives the agent maximum scope for the maximum duration. An OAuth-scoped token with cross-app access gives the agent a narrow scope for a narrow duration, mediated by an identity provider that can see the request and revoke it. The first option lets the agent do anything the human can do, on a schedule the agent chooses. The second option lets the agent do this specific thing, for this specific task, with an audit trail that survives the interaction.

The deeper claim is that identity and authority are joint design objects. Granular authority without a principal to attach it to is unenforceable. A principal without scope is just a louder user. The hard work is binding them: producing an agent-shaped identity with agent-shaped scopes that match the agent-shaped workflows the system actually runs.

## Sandboxing is product infrastructure

Code execution makes the boundary physical: when an agent runs commands, the question is no longer who it is but what its process can touch.

Fouad Matin's security guidance for coding agents at OpenAI keeps returning to four concrete controls: sandboxing, network restriction, privilege boundaries, and human review. Treat them as the default-on baseline, not the hardening you add after an incident — a capable system will sometimes be confused, manipulated, or overly eager, and each of the four bounds a different failure. Sandboxing contains a bad command, network restriction stops exfiltration and unexpected callouts, privilege boundaries cap what the agent can reach even when it is wrong, and a human review gate catches the trajectory the other three let through. Real safety has to live in the runtime and permission system, not only in the model.

Harshil Agrawal at Cloudflare makes the same case from the platform side. His talk on sandboxing AI-generated code argues that the sandbox is not an afterthought added once the model misbehaves; it is the substrate the agent runs on. If the agent is going to execute code, that execution needs to be bounded by default — network restricted, filesystem scoped, time-limited, resource-capped. The bounds are the product. Without them, the system is not running code on behalf of a user. It is granting the model an open shell on infrastructure.

Apple Private Cloud Compute, profiled by Jmo at CONFSEC, sits at the high end of this spectrum. The architecture is built around the principle that even Apple cannot see what runs inside a user's session. The cryptographic boundary is structural. That model is overkill for most product surfaces, but it makes the point in the strongest possible form: when the cost of getting trust wrong is unbounded, the boundary has to be designed-in rather than enforced socially. The right amount of sandbox for an enterprise coding agent is less than Apple's. It is also not zero.

Sandbox, least privilege, and auditability belong in the same category as evals, harnesses, and durable runtimes: they are product infrastructure, not security overhead. A team that treats them as the security team's problem will discover the boundary by getting it wrong in production. A team that treats them as part of the runtime spec has a chance of getting them right before that.

## Standardized protocols expand the attack surface

A common hope around the rise of the Model Context Protocol is that standardization will reduce the security problem by giving everyone a known interface to attack and defend. The actual pattern in the corpus is closer to the opposite. Easier interoperability means more tools can be exposed more quickly, which makes the questions of scope, mediation, review, and audit more urgent rather than less. The practical test: if adopting MCP raises the *number* of capabilities your agents can reach faster than your team can answer "who can call this, with what scope, and where is it logged?", standardization has expanded your attack surface, not shrunk it.

Tun Shwe at Lenses puts the bottom line plainly: "Your insecure MCP server won't survive production." The talk is essentially a tour of the ways MCP servers ship with assumptions that survive demo conditions and fail under real load. Authentication treated as a configuration option. Tool descriptions trusted as input. Servers exposed publicly because internal routing was the harder problem. None of these are MCP-specific bugs. They are the predictable consequences of pulling a wire protocol into a category — capability exposure — that has not had the security review the wire has.

Karan Sampath at Anthropic, working on enterprise MCP rollouts, names the structural answer from the governance side. "The really important thing for security teams ... is they need to establish a root of trust," he says. The enterprise version of this problem is not just performance, and it is not even just authentication. It is whether the security team can inspect the capability surface, reason about its risks, and produce a defensible record of what was allowed and why. A protocol that makes capability exposure faster does not lower that bar; it raises it.

David Mytton at Arcjet builds the same argument from outside-the-perimeter. His talk on defending sites from AI bots reads as a parallel chapter to the MCP discussion: as the ability to script automated interaction expands, the cost of identifying and bounding that interaction expands with it. The defender's surface is no smaller. It is larger, and the attacker now has standardized tools. It is tempting to assume the bot-defense problem and the MCP-exposure problem are separate teams' work; both are the same question — distinguishing authorized automated callers from unauthorized ones — and standardization arms the attacker on both sides of the perimeter at once.

Protocol standardization expands the attack surface if governance lags. It is not an argument against standardization. It is an argument that standardization is a forcing function for governance work that has to happen in parallel, not afterwards.

## Enterprise MCP rolls up to gateways and a root of trust

Once enterprise teams take the MCP security problem seriously, the architecture they reach for converges on a recognizable shape — and the convergence is specific enough to use as a checklist. There is a gateway. There is a policy plane. There is a registry of blessed servers. There is a permissions model that knows about identities, tools, and approval requirements. There is an audit log that records what each agent invocation actually did. If you are standing up enterprise MCP and any one of those five is missing, that gap is the likeliest place for the boundary to fail first. The shape looks a lot like the API-gateway and IAM patterns that mature enterprise SaaS settled into a decade earlier — applied to a new category of capability, which means the teams that already own those patterns are the ones to build the agent version, not a greenfield security project.

Sampath's enterprise MCP work names the architectural conclusion directly. The root of trust is established at the platform, not at the individual tool. Servers are reviewed before they are allowed in the corporate environment. Tools are scoped to the principals that should be able to call them. Audit is built in at the gateway layer rather than tacked onto each integration. This is the enterprise equivalent of moving from "anyone can install any app" to "we have an internal app store with security review." Unglamorous, and exactly what lets the technology be used inside a regulated enterprise at all.

Sam Morrow's GitHub MCP work shows the same shape at production scale. As part of scaling MCP for GitHub's customers, the team filtered tools by PAT (personal access token) scopes — so an agent invoking a GitHub MCP server only sees the tools its token actually authorizes — and used step-up OAuth to request additional privileges only when needed, rather than pre-granting them. The effect is that the agent's effective capability surface is dynamically bounded by the identity it is acting under and the task it is currently performing. Capability follows authority, not the other way around.

These two cases together make the architectural argument concrete. Enterprise MCP adoption pushes toward gateways, blessed platforms, and a root of trust. Naming the pattern matters because it changes what teams build first. Instead of "ship one MCP server and add another, and another," the platform shape becomes the first deliverable. Individual servers are then deployed against a structure that knows how to govern them.

## Per-tool OAuth flows are a governance problem

Most current MCP and agent integrations require the user to authorize each tool separately. The agent wants to talk to email. Consent flow. The agent wants to talk to calendar. Consent flow. The agent wants to talk to the document store. Consent flow. To the operator, this is a paper-cut sequence of OAuth dialogs. To the security team, it is far worse: it is invisible. Each consent grant happens between the user and the third-party tool, mediated by an identity layer the enterprise may or may not control. The IT organization has no central record of what authority the user just delegated, to which tools, under what scopes, with what expiry.

Galow's cross-app access work at WorkOS is one of the cleanest framings of the structural answer. The identity provider becomes the bridge between MCP clients and servers, so that credentials can be obtained without repeated manual consent flows *and* so that the enterprise has a single place to see — and revoke — what was delegated. That second part is the load-bearing one. A faster consent flow with no visibility is not progress. A consent flow that produces a visible, revocable audit trail at the identity layer is.

Hanson's OAuth argument fits into the same shape. The right pattern for agent credentials is short-lived, scoped tokens issued through a flow the identity provider can audit — not standing keys, not impersonation, not human credentials borrowed for agent work. Morrow's step-up OAuth on GitHub is the same pattern at the tool layer: the agent never holds more authority than it currently needs, and any escalation goes through a flow that produces a record.

Repeated per-tool OAuth flows are not just annoying; they are a governance and IT visibility problem. The fix is structural — push the trust bridge into the identity provider — and the architectural payoff is the same as in the gateway argument: the enterprise can answer the basic auditing questions ("who delegated what, to whom, when, and for how long?") because the system was built to capture them, not because someone reconstructed them after the fact.

## Audit as part of the trust model

Mark Myshatyn at Los Alamos National Lab presented on government agents at one of the AI Engineer events. The public-sector case makes the constraints sharper than any enterprise version, because failure carries legal consequences rather than embarrassment. When the user is a federal employee, the data is classified, and the action might be regulated under specific statutes, the identity, authority, and audit questions stop being best practices and become legal requirements. The agent has to act under a real principal. The scope has to be documented. The audit trail has to survive review. The boundaries have to be enforceable, not aspirational. Most enterprises are not national labs, but most of these constraints are the ones other regulated industries are now discovering for themselves.

The observability argument from the previous chapter returns here in a stricter form. Audit trails, trajectory views, approval logs, and replayable histories are not just debugging conveniences. They are part of the trust model. If a system drafts a return, assembles a legal research memo, or performs a sensitive code change, an institution needs to be able to answer basic questions afterward: who authorized this path, what evidence was consulted, what tools were used, what warnings appeared, and where a human judgment entered or failed to enter. Without those answers the institution cannot certify the work. Without certification, the work cannot ship.

This is also where the manuscript has to stay honest. Conference talks are especially good at surfacing strong patterns, field reports, and architecture instincts. They are weaker as proof of universal effectiveness. So the responsible claim here is not that the industry has solved trustworthy delegation. It has not. The more grounded claim is that serious teams keep discovering the same constraints: narrow scopes, explicit identities, mediated tools, sandboxes, review points, and inspectable histories.

A machine colleague is not trustworthy because it sounds careful. It is trustworthy, if at all, because its power has shape.

The next chapter takes the chapter-7 frame and stress-tests it under realtime conditions, where the cost of getting bounded authority wrong becomes audible to the user inside a single conversation. But the move there only works because the chapter behind it has named the shape of power, the substrate of identity, and the architecture of audit. Those are the pieces. The next chapter is what happens when they have to ship under a 200-millisecond clock.

## What to do with this

- Audit your agents for standing credentials. Any long-lived API key, service-account token, or "borrowed" personal access token is the agent inheriting a human's full authority with no scope to revoke and no expiry that maps to the task. Replace them with short-lived, scoped OAuth tokens bound to a session, a user, and a set of scopes — the pattern Hanson at Keycard argues is the right shape for delegation, not just a protocol choice.

- Make the agent a real principal in your identity provider, not a passenger on a human's credential. Auth0's Riley and Galan frame this as "we authorize agents, MCP servers" — once the agent has its own scopes, lifetime, audit footprint, and revocation path, an agent authenticated as a blurry extension of a human stops being a delegation and starts being impersonation.

- When scoping agent permissions, ask "minimum access for which stage of the workflow?" — not just "minimum access." A scope that is harmless in isolation can compound across retrieval, reasoning, persistence, and retries, so least privilege for agents has to be bound to the workflow stage, not granted once for the whole run.

- Make the sandbox the default substrate for any code-executing agent: network restricted, filesystem scoped, time-limited, resource-capped (Agrawal/Cloudflare), plus Matin's four controls — sandboxing, network restriction, privilege boundaries, and human review. Without those bounds you are not running code on behalf of a user; you are granting the model an open shell on your infrastructure.

- Before standing up enterprise MCP, build the platform shape first, not one server at a time: a gateway, a policy plane, a registry of blessed servers reviewed before they enter the environment, a permissions model over identities and tools, and an audit log at the gateway layer. Sampath's rule is to establish the root of trust at the platform, not the individual tool — treat it like an internal app store with security review.

- Filter tools by the caller's token scopes and use step-up OAuth to request extra privilege only when needed, the way Morrow's team did for GitHub's MCP — so the agent only ever sees the tools its token authorizes and any escalation produces a record. And push per-tool consent through the identity provider as a trust bridge (Galow/WorkOS) so IT keeps one revocable, auditable record of who delegated what, to whom, when, and for how long.

---

# Chapter 8 — Realtime, Voice, and the Cost of Being Interruptible

Text chat flatters AI systems. Voice does not.

In text, users tolerate delay, awkwardness, and weak handoffs because the interface gives the system time to recover. In spoken interaction, every defect becomes audible. If the model pauses too long, it sounds confused. If a tool call stalls, it sounds incompetent. If the system interrupts badly or loses the thread after a clarification, it stops feeling like an unfinished prototype and starts feeling like a bad conversational partner.

That asymmetry is why voice deserves a chapter, not a section. Realtime interaction pressure-tests almost every earlier claim in the book. Weak context becomes an audible loss of thread. Brittle runtime design becomes broken interruption handling. Sloppy authority boundaries become dangerous casual approvals. Slow tools become conversational incompetence. Voice does not introduce a new thesis; it makes the existing one harder to hide from.

## A stress test the model alone cannot pass

The natural assumption among engineers stepping into voice for the first time is that the model is the bottleneck. If the speech-to-speech system feels stiff, the obvious culprit is the LLM, so the first instinct is to swap in a faster or smarter model and expect the experience to converge to a conversation. It rarely does, and the reason is measurable: profile a stiff voice agent end to end and the time is almost never in inference — it is in the tool call, the turn-detection delay, and the orchestration between stages. Upgrading the model leaves all three untouched.

The practitioners building production voice systems do not see it that way anymore. Sean DuBois at OpenAI and Kwindla Hultman Kramer at Daily titled their joint talk "Your realtime AI is ngmi" for a reason: the gap between current voice agents and natural conversation, in their experience, is dominated by problems no model improvement will close. Tool-call variance. Overlap handling. Interruption recovery. Turn budgets. None of those are model-quality problems. They are systems problems with an architectural shape.

Neil Zeghidour at Gradium AI makes the strongest version of this argument. "The main bottleneck is becoming the tool call," he says, after walking through what each layer of a voice agent now costs. Speech recognition is fast. The model itself is fast. Speech synthesis is fast. What is not fast — and worse, what is not *predictable* — is the call out to a function or an external service that the agent has to make in the middle of a sentence. "You have a tool call or open router that is going to have a latency between 500 milliseconds and 4 seconds," he notes. The variance is more painful than the average. A 4-second pause in conversation is not a slow response. It is a dead line.

Dippu Singh frames the same problem from inside the contact-center stack. "Processing real-time voice data is an engineering minefield of latency, accents, and interruptions," he writes, and *minefield* is the precise word — the failures do not announce themselves on the way in. They compound. An early error in the pipeline — a misread turn boundary, a misheard entity — does not surface where it happened; it propagates into routing, retrieval, and the final response, so the visible symptom is a confidently wrong answer several steps downstream of the actual fault. The practical consequence is that voice systems have to be traced per turn, not per session: log the turn boundary, the transcript, the retrieved record, and the response together, so that when an answer is wrong you can walk back to the stage that caused it instead of debugging the last step alone.

So the chapter that follows treats voice the way the rest of the book has treated coding, evaluation, context, and runtime: as a problem of the loop around the model, not of the model itself. Concretely, that means the budget to defend is the end-to-end turn — from the moment the user stops talking to the moment they hear a reply — and the things that defend it are orchestration, masking, and turn-taking architecture, in that order. The sections below take them one at a time.

## Latency is a budget across the whole stack

Zeghidour's working number for natural conversation is precise. "The entire stack of understanding, producing an answer, and pronouncing it" needs to land "around 200 milliseconds." That is the budget. Not the recognition latency. Not the inference latency. The end-to-end turn budget — from the user finishing their sentence to the user hearing the system's reply.

Use Zeghidour's number as a pass/fail line, not an aspiration. Instrument end-to-end turn latency from end-of-utterance to first audio packet, set ~200ms as the target and a sub-second P95 (Kramer's contact-center threshold, later in this chapter) as the line you must not cross, and treat every turn past it as a defect to triage rather than a number to admire. The trap is optimizing the wrong number — shaving inference milliseconds, which were already fast, while the turn budget blows out on a tool call you never put on the same dashboard. Put the end-to-end turn and the tool-call tail on one chart, because that is the only place the real failure shows up.

Inside that budget, the system has to do at least: detect end-of-utterance, transcribe the audio, build the context, run inference, possibly call one or more tools, generate the response text, synthesize it as audio, and start streaming it. The components that have been heavily optimized — speech recognition, model inference, text-to-speech — are no longer the dominant cost. Mark Backman's Daily workshop on realtime voice walks through this in production-grade detail: the engineering wins of the last two years have come almost entirely from squeezing variance out of orchestration, not from making any single component faster on the median.

Kwindla Hultman Kramer's Pipecat Cloud work makes the same point as infrastructure rather than commentary, and he is specific about the number to watch. The metric is not average — it is the tail. "In most cases in a voice AI conversation, you care a lot if your P95 goes up above 800, 900, 1,000 milliseconds for the entire voice-to-voice response chain," he says, "and all the little inference calls you make as part of that have to be much faster than that by definition." That gives you a hard budgeting rule: set a P95 ceiling on the end-to-end chain — Kramer's is sub-second — and then back out per-stage deadlines that sum to it with headroom, so every component knows its slice. The contrast he draws is with a plain HTTP service, where a P95 of 1500ms or 2000ms is fine; in voice it is a failed conversation. And because any individual call can still miss its deadline, the orchestration is designed to handle the lateness — masking it or routing around it — rather than trusting the late component to recover.

This is the inversion the chapter rests on, and it changes what you optimize for. In a text agent, latency is a property you tune after the system works; you can ship with a slow tool and fix it later. In a voice agent, a missed budget is a broken turn, so latency is a constraint you design against from the first sketch — which means deciding up front, per operation, whether it fits the budget or needs masking. The practical drop-out: any tool call whose P95 sits inside the budget can stay synchronous; any whose P95 can exceed it must either move to a parallel/background call or get a filler in front of it. You make that call per tool, at design time, not after a user complains the system feels frozen.

## Wrap, don't rebuild

The expensive wrong turn in a voice project is to throw away the working chat agent and rebuild on a speech-native stack, on the theory that voice is different enough to need its own architecture. It is the wrong turn because the rebuild discards exactly the parts that took the longest to get right — the tools, the evals, the state model, the permission and retry logic — none of which are voice-specific. You spend months reaching the reliability you already had, minus the months you spent reaching it the first time. The tell that you are about to make this mistake is the moment you start specifying a new tool layer for the voice system instead of pointing the voice layer at the tools the chat agent already calls.

Luke Harries at ElevenLabs makes the case for the opposite move with one of the more honest sentences in the corpus. "I've already got my agent. I spent loads of time doing the evals," he says, framing the eval infrastructure not as something the voice migration can throw away but as the asset that justifies keeping the existing architecture. The right move, in his framing, is to "wrap it up into its own first-class primitive" — to add a voice engine *to* the agent rather than rebuild *as* a voice agent.

The reasoning is sharper than it first appears. The hard parts of a production agent — the tools, the evals, the durable state, the permission model, the retry logic, the observability — are not voice-specific. They are agent-specific. Throwing them away to start from a speech-native stack means throwing away the work that made the system trustworthy in the first place. As Harries notes, "your chat agent actually normally does the majority of tool calling." The voice layer's job is to make that agent audible. The agent's job remains the same.

Thor Schaeff's ElevenLabs workshop on building conversational AI agents is essentially an operational version of the same argument, and it names what the voice layer actually has to add. Turn detection decides when the user has stopped talking, so the agent knows when to run — get this wrong long and the system feels slow, wrong short and it interrupts. Barge-in handling lets the user cut the agent off mid-sentence, which means the speech output must be cancelable on a moment's notice, not fire-and-forget. Partial audio streaming plays the response as it generates instead of after, which is the perceived-latency win from the TTS section. A speech model supplies recognition and synthesis. All four sit in front of the existing agent; none of them touch its tools, evals, or state. The voice layer is an adapter, not a replacement.

This pattern shows up across the cluster. Brooke Hopkins at Coval frames her voice work as "from self-driving to autonomous voice agents," and the analogy is not casual: she treats the voice runtime the way an autonomy stack treats the perception and planning layers — as composable subsystems with their own contracts, not as a monolithic rewrite of the underlying control system. The operational test: if adding voice forces you to touch the agent's tools, evals, or state model, you are rebuilding when you should be wrapping. Draw the seam at the interface, give the voice layer (turn detection, barge-in, audio streaming, speech model) its own contract, and leave the orchestration untouched. The agent persists across the migration. The interface does not.

This is where the architectural move the book has advocated since Chapter 3 — keep the orchestration durable, change the surfaces around it — pays a concrete dividend. You can tell which kind of agent you have by one question: is your business logic in the orchestration layer, or in the prompts and the UI? An agent whose tools, evals, and state live in a durable orchestration takes voice as a new surface — wrap it and ship. An agent that is a thin wrapper, with the logic smeared across prompt strings and front-end glue, has nothing clean to wrap, so adding voice means rebuilding the orchestration it never had. The voice migration is the bill for that earlier shortcut, and it comes due all at once.

## Half-duplex is the silent architectural ceiling

Even with the latency budget defended, one ceiling remains that no amount of orchestration speed can lift, because it is baked into the model's architecture: whether the system can listen and speak at the same time. This is the constraint to check before you promise natural conversation, because it decides what kinds of conversation are possible at all.

Most production speech-to-speech systems are half-duplex. They can listen, or they can speak, but not both at the same time. As Zeghidour puts it directly: "The model is either listening or it's speaking." You can detect which kind you have with one test: while the agent is speaking, say "mhm" or start a short overlapping interjection. A half-duplex system either ignores it entirely or treats it as a barge-in and stops; it has no third option, because it cannot process incoming audio while its own output is playing. That single architectural choice caps every interaction that does not happen strictly one turn at a time — and Zeghidour's point is that natural conversation is shot through with overlap. In Japanese, where back-channeling is a politeness norm, he notes overlap runs "up to 20% of the time." A model that cannot be talked over during that 20% is not having the conversation the user is having.

This is why half-duplex should steer your product scope, not just your model choice. Overlap and back-channeling are first-class features of human dialogue, and a half-duplex model cannot do either — so the honest move is to design the interaction so it does not need them. Structured, turn-based exchanges survive half-duplex cleanly: IVR menus, customer-support flows, form-filling, anything where one party clearly holds the turn. Open-ended companion or brainstorming chat does not, because the user will instinctively interject and the model will read every interjection as a barge-in. Match the duplex capability to the use case up front; do not promise a flowing two-way conversation on an architecture that can only take turns.

Tom Shapland's LiveKit talk "Why ChatGPT Keeps Interrupting You" is a case study in the specific failure mode this produces. The system has no way to lean in without taking the turn and no way to wait without going silent, so every micro-decision collapses into one binary: am I listening, or am I speaking? Zeghidour demonstrates the result live — when a user back-channels at a half-duplex agent, the agent reads each "mhm" as a turn handoff and keeps stopping and restarting, until the human is reduced to telling it "please stop interrupting." That is the diagnostic to watch for in your own logs: a spike in interruption events that correlate with user filler words rather than real turn-taking means the system is misclassifying back-channels as barge-ins. The fix is not a better silence threshold; it is either a model that can distinguish a back-channel from a turn grab, or a use case that does not produce back-channels.

The instinct of many teams is to patch this in product. Add a *wait* button. Add a tone-detection heuristic. Add a longer silence threshold. The trap: each patch helps marginally and is paid for in worse responsiveness somewhere else — a longer silence threshold buys cleaner turn-taking at the cost of a slower system. Treat these as symptoms, not fixes. When you find yourself stacking silence-timing tweaks, the right read is that the half-duplex constraint is propagating, and the lever is the architecture, not the timing.

The cleanest long-term answer is full-duplex architecture — systems that can simultaneously listen and speak. Zeghidour's own Moshi work is one of the few production-scale demonstrations. As he is the first to point out, full-duplex models pay for the architectural change at the agent layer: the model can do conversational overlap, but it is materially weaker at the actual reasoning. The trade is real. The half-duplex world is a workplace; the full-duplex world is a research frontier.

So make the duplex decision early, at the same point you pick the speech model, because it is a property of that model and not something you can add later in product. Concretely: write down whether your use case needs overlap. If it does not — IVR, support triage, structured tasks — a half-duplex model is the right call and you should stop trying to make it feel full-duplex with silence-timing hacks. If it does, budget for a full-duplex model and accept the reasoning-quality trade that comes with it. The one option that does not exist is a half-duplex model that behaves full-duplex once you tune the timing well enough.

## Latency must be masked, not just minimized

The 200-millisecond budget is the target for the turn itself. The problem is the tool call inside it. Zeghidour's measured range for a tool call or open router is 500 milliseconds to 4 seconds, and the variance is the part that bites: you can sometimes optimize the median down, but the worst case is set by an external service you do not control, so on a bad request the turn budget is already blown before your own code runs. This is the gap that makes a second technique necessary alongside optimization. You shrink what you can, and for the rest — the calls whose tail can exceed the budget no matter how you tune them — you need a way to keep the conversation alive through the wait.

The systems that ship anyway do it with latency masking: a layer that emits low-latency conversational audio — an acknowledgement, a clarifying question, a holding phrase — to cover the gap while a slow operation runs underneath. It is a distinct layer with its own job, not a property of any one component, which is why it has to be designed in rather than discovered late.

Zeghidour describes the pattern as the agent producing fillers and partial answers: "While it waits for getting the result back, it can keep the conversation going." A short acknowledgement here. A clarifying question there. A *one moment* that buys a second or two of tool time without leaving the user staring into silence.

This is not animation. It is conversational scaffolding. The fillers are doing structural work — they preserve the user's sense of turn, they signal that the system is alive, and they create slack the orchestration needs to handle variance. The decision rule: any operation whose latency variance can exceed the turn budget needs a masking strategy before it ships — a short acknowledgement or clarifying question emitted the instant the slow call starts, not after it stalls. Done well, a 3-second tool call disappears into the rhythm of the exchange. Done poorly, the same call sounds like a freeze.

Harries makes the same point from the architecture side. The voice layer needs the ability to interleave low-latency speech with the agent's longer-running operations, which is why he treats the voice engine as a first-class primitive with its own state — not as a streaming pipe attached to the agent's output. When the agent is busy, the voice layer is not idle. It is doing the conversational equivalent of nodding.

Suman Debnath's VoiceVision RAG work at AWS surfaces the same pattern at a different layer, and it generalizes into a rule: any slow subsystem the conversation depends on — a vision model reading a document, a retrieval step, a long tool call — should run as a background task with the voice thread held in front of it, never as a blocking step the user waits through in silence. The user does not care that a document was being read in the background. They care that the conversation did not stall.

You can see whether masking was designed in or bolted on by where the filler decision lives. Designed in, the masking is part of the tool-call contract: each slow tool declares its expected latency, and the orchestration emits the holding phrase the instant it dispatches the call — the filler and the call start together. Bolted on, the filler is a timeout fallback that fires only after the system has already gone silent for a second or two, so the user has already heard the freeze before the cover arrives. The first is reliability infrastructure; the second is an apology. If your masking is implemented as a silence-detector watchdog rather than as part of how you dispatch slow work, you built the second one.

Latency masking belongs in the same architectural category as evals, harnesses, and durable runtimes — and it has the same failure mode as a leaky abstraction: it is cheap to abuse. The line to watch is whether the filler is covering real work or covering its absence. A *one moment* that resolves into an answer is scaffolding; a *one moment* repeated with no payoff teaches the user the system is lying about progress, and they stop trusting every acknowledgement after it. Mask only operations that are actually running, and measure the gap between filler emitted and result delivered the way you would measure a tail latency. It is the runtime cost of being interruptible.

## TTS architecture is converging on LLM architecture

The clearest sign that voice is the same engineering problem in a different medium is that speech generation itself has started to adopt the LLM's architecture: leading text-to-speech systems have converged on an autoregressive decoder backbone, streaming token-by-token under a first-packet-latency budget. The practical consequence is concrete: do not wait for the full audio to render before you play it. Humeau is explicit about the mechanism — "as soon as you have the first audio packets, you start to voice them out. This way, the perceived latency is lower." Wire the TTS the same way you wire the LLM: emit the first packet the moment it exists, and pipeline synthesis against generation so the speech model is decoding the start of the sentence while the LLM is still producing the end of it. The win is in perceived latency, not median throughput, and it is unavailable to any TTS you treat as a render-then-play black box.

Samuel Humeau at Mistral, whose work on TTS-as-LLM is the chapter's reference for this thread, puts the convergence plainly. "Pretty much everybody is using an auto-regressive decoder backbone," he says, describing the architectural shape the leading text-to-speech systems have settled into. That shape is what makes the streaming move above possible at all: an autoregressive backbone emits audio tokens left-to-right, so you can start voicing the first tokens before the last ones exist — the same reason an LLM can stream text. So when you evaluate a TTS vendor or model, the question that predicts whether you can hit a conversational budget is not the MOS quality score, it is whether the model exposes a streaming, token-by-token decode you can start playing on the first packet. A high-quality model that only returns a finished audio file cannot be made conversational by optimization; the latency floor is the full render time, every turn.

The convergence is not only about streaming; it reaches the conditioning interface too. Because the TTS shares the LLM's autoregressive shape, you can feed it the LLM's output token-by-token as it generates, rather than waiting for a finished string and re-prompting the synthesizer. As Humeau frames the target system: as soon as you have the first token of the LLM, the machine starts to speak. That tightens the LLM-to-TTS seam from a handoff into a pipeline — and it is the same conditioning-on-the-running-stream discipline the chat side already uses, now applied one layer down.

Humeau also identifies the use case driving the convergence. "The king use case for text-to-speech is its usage within agents," he says — and that has a direct selection consequence. If your TTS is going inside an agent, the spec that matters is not narration quality over a paragraph; it is two things the agentic loop demands and standalone audio production never tested: a low and stable time-to-first-audio-packet, and clean interruptibility — the ability to cut the current utterance mid-word the instant the user barges in, without a tail of buffered audio that keeps talking over them. A model tuned for audiobook-grade prosody can fail both. When you benchmark a TTS for agent use, measure time-to-first-packet and stop-on-barge-in latency before you score how it sounds.

Humeau is specific about where his own released model departs from the pattern, and the distinction is the one you should build against. The backbone is autoregressive and streams frame after frame — that is the part that has converged across labs. But the per-frame stage is not autoregressive: Mistral generates each frame's "37 tokens at once using a diffusion model" rather than one sample after another. So the interface you can portably rely on is the streaming backbone — frames emitted left-to-right that you start playing on arrival. The internal frame generator (autoregressive, diffusion, flow-matching) is a vendor implementation detail that changes between models. Design your pipeline to consume a stream of frames; do not couple your orchestration to how any one model produces a frame, because that is exactly the layer where the field has not converged.

The practical upshot of the convergence is that you can carry the chat-side serving discipline straight into the TTS layer instead of treating speech as a separate craft. The same three levers apply: stream the first packet rather than blocking on the full response, condition the model on the running text rather than re-prompting it, and budget time-to-first-audio rather than total render time. A team that already knows how to serve an LLM under a latency budget already knows most of how to serve a TTS under one — which is the same reason the wrap-don't-rebuild move pays off one layer up.

## Robotics: the same constraints at higher stakes

Robotics runs the same playbook against a harder clock. The mapping that follows is an authorial analogy, not a corpus finding — the robotics talks in the cluster (GR00T N1 humanoid foundation models, Waymo's EMMA, Physical Intelligence's "why now?" framing) are named here as illustrations, not anchored claims — but it holds up primitive for primitive. The end-to-end turn budget becomes the perception-to-action loop: instrument the whole loop, not the policy network, because the time is almost never in inference. The half-duplex ceiling becomes one-action-at-a-time: a robot mid-motion cannot start a new commitment any more than a half-duplex model can speak and listen at once. And latency masking becomes motion-plan continuity — where a voice agent emits a filler to hold the turn, a robot holds a smooth interpolated trajectory through a slow planning step so it never freezes mid-air. What changes is the budget size and the cost of blowing it: a voice agent's missed deadline, set by conversational comfort, is an awkward pause; a robot's, set by physics, is a collision — and there is no graceful-degradation path, because you cannot mask a blown perception-to-action loop with a filler.

That is the boundary worth carrying back, and it is the reason the coda matters to a voice engineer and not just a roboticist: robotics shows where latency masking has a hard limit. Masking works only when the slow operation is reversible and the user just needs the conversation to stay alive. Remove the option to stall and the lever disappears — the only moves left are to make the loop genuinely fast or to refuse the action. So if your voice system is ever doing something it cannot take back during a masked pause — charging a card, sending a message, committing a transaction — you are in the robotics regime, and a filler is not a safe substitute for finishing the work. A voice agent that picks the wrong filler recovers on the next turn; a commitment that cannot be un-committed sets the agency dial lower the more a mistake costs to undo — the same risk-versus-irreversibility logic that governs how much autonomy any high-stakes agent is given.

## What voice confirms about the book's frame

If you build one thing out of this chapter, build the per-turn trace. Voice failures compound silently — a misread end-of-turn or a misheard entity does not surface where it happened; it propagates into routing and retrieval and shows up as a confidently wrong answer several steps downstream. The only way to walk an error back to its stage is to log each turn as a unit: the turn boundary, the transcript, the retrieved record, and the response, tied together. Trace per turn, not per session. Without it you debug the last step alone and never find the stage that actually broke.

Run your system against the chapter's checks before you ship it. Is your latency metric the end-to-end turn, end-of-utterance to first audio packet, with ~200ms as the line — or are you still reporting inference speed? Is your tool-call latency budgeted on its worst case in the 500ms–4s range, or on its average? Does every operation whose variance can blow the turn budget emit a filler the instant the slow call starts? Does your TTS stream and stop cleanly on barge-in, or render-then-play? Did adding voice leave the agent's tools, evals, and state untouched, or did you rebuild? Each question targets a failure that text mostly hid and voice makes audible; a "no" to any of them is where the experience will break.

Realtime pressure also gives the earlier chapters a sharper test. The harness from Chapter 3 is what you wrap rather than rebuild when voice arrives — which is why a team with a strong harness has already done most of the voice work, and a thin chat wrapper has to start over. The evals from Chapter 4 are the asset Harries refuses to throw away in the migration. The context-assembly discipline from Chapter 5 is what the per-turn trace logs. The durable runtime from Chapter 6 is what lets a slow tool call run in the background while the masking layer holds the turn. None of those are voice features; voice is just where their absence becomes audible within one turn instead of over a session.

The next chapter widens out to the organizational consequence. The voice work in this chapter does not live in a model team's backlog — it is orchestration, tracing, masking, and turn-taking infrastructure, owned by whoever owns the loop around the model. A team organized to fine-tune models will not staff the per-turn trace or the masking layer, because those are not model problems. Chapter 9 is about what the team that does ship dependable AI is actually structured to build.

The last word in the voice cluster belongs to Zeghidour, who has been honest enough throughout his talk to keep saying that there is more to do. "The last mile is going to be the most difficult to solve," he says, near the end. The chapter would not put it differently. The first mile of voice was a model problem — speech recognition, synthesis, and the language model itself, all now fast. The last mile is three concrete pieces of infrastructure, and each has a measurable definition of done. The orchestration holds the end-to-end turn budget: instrument end-of-utterance to first audio packet, hold it near 200ms, and budget tool calls on their 4-second worst case, not their average. The masking layer keeps the conversation alive through that variance: every operation that can exceed the budget emits a filler the instant it starts, and you measure the gap between filler emitted and result delivered like a tail latency. The duplex ceiling decides whether overlap is possible at all: half-duplex today, full-duplex when you are willing to trade reasoning quality for it. The first mile is mostly solved. These three are the work, and they are the same work the rest of this book has been describing.

## What to do with this

- Measure the end-to-end turn budget, not component speed. Instrument latency from end-of-utterance to first audio packet and treat ~200ms as the pass/fail line; if you're shaving inference milliseconds while a tool call blows the budget, you're optimizing the wrong number.
- Wrap your existing agent; don't rebuild it as voice-native. Add the voice layer (turn detection, barge-in, partial audio streaming, speech model) as a first-class primitive over the agent you already evaluated — and use this test: if adding voice forces you to touch the agent's tools, evals, or state model, you've crossed from wrapping into rebuilding.
- Profile your tool-call latency *variance*, not its average. A call that averages fine but spikes to 4 seconds is the real failure; Zeghidour's "500 milliseconds and 4 seconds" range is what kills conversations, because a 4-second pause is a dead line, not a slow response.
- Build a latency-masking layer for any operation whose variance can exceed the turn budget. Emit a short acknowledgement or clarifying question the instant the slow call starts — not after it stalls — so a 3-second tool call disappears into the rhythm instead of sounding like a freeze. Design it in as reliability infrastructure, not as a bolted-on afterthought.
- Recognize half-duplex as an architectural ceiling, not a UX bug. When you find yourself stacking silence-timing tweaks and *wait* buttons, stop — each patch is paid for in worse responsiveness elsewhere; the lever is the architecture, and full-duplex buys conversational overlap at the cost of weaker reasoning, so make the trade deliberately.
- Treat TTS as an LLM-shaped system. Reuse the streaming, first-packet-latency, and early-emit discipline from the chat side — stream the first audio packet before the full response is generated so perceived latency drops — because leading TTS work has converged on autoregressive decoder backbones optimized for embedding in agents.

---

# Chapter 9 — The AI-Native Organization

Most AI adoption stories are too small to be the real story. A few engineers get faster. Support summarizes tickets more quickly. A product manager drafts specs in half the time. Those are real gains, and they are also, every one of them, individual gains. Add them up and you have a company that uses AI. You do not yet have an AI-native organization.

The difference shows up on a Monday morning. Picture a company that has already moved past casual adoption. Over the weekend, agents opened pull requests. Product generated three new onboarding flows. Support drafted fixes for a backlog of tickets. Internal automations touched the billing system, the CRM, and the docs. Everyone — and everything — was productive. And the organization walks in on Monday to discover that its problem is no longer a shortage of output. It is a shortage of coherence.

That is the shift this chapter is about. AI does not simply make an organization faster. It moves where the scarce thing lives. When execution gets cheap, the bottleneck stops being production and becomes judgment, prioritization, review, and the design of throughput itself. The companies that win the next decade will not be the ones that bought the most seats. They will be the ones that redesigned the operating model so that cheap generation turns into trusted work instead of expensive noise.

## AI-native is an operating model, not a purchase

The cleanest way to see the gap between "uses AI" and "AI-native" is to look at what happens at the threshold of full adoption. Dan Shipper, building the AI-native company Every, puts the discontinuity in numbers: "There is a 10x difference between an organization where 90% of engineers use AI versus one where 100% do." The claim is a deliberate provocation, not a measured figure — the steepness is the point, and the more sober large-scale evidence comes a few paragraphs on. The last ten percent is not a rounding error. It is the difference between AI as a personal productivity tool and AI as a medium the whole organization works in.

The reason is that partial adoption keeps the old workflows intact. If nine in ten engineers use agents but the tenth does not, every process still has to accommodate the human-only path — the review that assumes a person wrote the code, the handoff that assumes a person is on the other end, the planning that assumes work moves at human speed. The workflow cannot be rebuilt around delegation until delegation is universal, which makes the holdout the constraint, not a rounding error: as long as one path assumes human-speed work, you cannot rebuild the surrounding process around delegation, so the gain from the other ninety percent is capped at faster people rather than a faster company. The lever, then, is not buying the ninety-first seat but closing the last human-only path. At ninety percent, you have faster people inside an unchanged company. At a hundred, the company itself can change shape.

The industry-scale evidence points the same way. Barr Yaron's 2025 AI Engineering Report found that roughly eighty percent of respondents say LLMs are working well at work — and yet the reports that read as transformational are not the ones with the highest adoption numbers but the ones in the "from hype to habit" cohort, the teams still shipping a roadmap while rebuilding how it gets built. The arc they describe is consistent: the early win is individual speed, and the durable win arrives only when the surrounding work — review, planning, handoff — is rebuilt to assume that speed. High adoption is the entry ticket, not the finish line. "We rolled out AI" buys the ticket; "we became AI-native" is the redesign, and only the redesign produces the order-of-magnitude difference Shipper is pointing at.

## Cheaper execution moves scarcity up the stack

When code gets cheap, judgment gets expensive — Chapter 1 made that a claim about delegation and Chapter 2 a claim about taste. At the organizational scale it becomes a question with an operational answer: which queue is actually backing up? If your engineers are faster but nothing ships sooner, the constraint has already moved off production and onto something downstream — deciding what to build, or confirming that what got built is correct — and the next several sections are about finding which.

When a single engineer can direct several agents at once, the constraint stops being how fast the team can produce and starts being how fast the team can decide what is worth producing and confirm that what got produced is correct. Justin Reock, working on engineering leadership at DX, frames the leadership job in exactly these terms: the manager's role shifts from allocating production capacity — which is suddenly abundant — to allocating judgment and attention, which stay scarce. The practical test for a manager is to ask which scarce resource each ritual rations. A standup that reports how much got produced is rationing the abundant thing; a standup that surfaces which decisions are unmade and which output is waiting on review is rationing the scarce one. The org chart was built to ration the wrong resource — and the wrong choice is to keep optimizing throughput when the queue that is actually backing up is judgment.

Here the manuscript has to stay honest, because this is precisely the place where AI hype outruns the evidence. The large-scale studies are more sober than the demos. Yegor Denisov-Blanch's Stanford research, drawn from data on a hundred thousand–plus developers, finds that AI's productivity effect is real but uneven — strong in some task types and codebases, near zero or negative in others, and reliably overstated when teams measure activity instead of outcomes. Some AI-generated work creates rework that quietly eats the gain. The responsible reading is not "AI makes everyone faster." It is "AI changes the distribution of where time goes, and a naive measure will misread motion as progress."

That nuance matters for org design because the wrong metric, applied to cheap execution, actively destroys value. Nick Arcolano's analysis at Jellyfish, built on a dataset of some twenty million pull requests, shows the failure mode at scale: output volume rises, the activity dashboards light up green, and the actual constraint — whether the organization can review, integrate, and trust all that output — goes unmeasured until it breaks. The failure to name is the green dashboard: a count of PRs opened, commits, or lines generated is an activity metric, and in a world where artifacts are cheap, an activity metric measures motion, not progress. The fix is to instrument the outcome instead — rework rate, the share of generated work that ships unreverted, time spent in the review queue — so the metric tracks the resource that actually moved. The scarce resource moved; the measurement did not.

## The same target looks different at different scales

The shift from "uses AI" to "AI-native" is one destination, but the road to it is not the same for a fifteen-person team and a thousand-person enterprise. The blockers, the failure modes, and the order of operations diverge sharply with size. The three-scale split that follows — seed, scale, enterprise — is an organizing device rather than a taxonomy the corpus hands us: only the scale-stage case (Bloomberg) rests on an anchored source, and the boundaries are illustrative, not measured. Read it as a way to locate your own bottleneck, not as a validated maturity model. A useful diagnostic is to stop asking "how advanced are we?" and start asking "which scarce resource is our current bottleneck rationing?" — because the answer points at different work in different places.

For a seed-stage team of five to fifteen people, the biggest risk is not ungoverned sprawl. It is the false plateau. At this scale everyone is aligned by proximity — the same room, the same Slack channel — so the ambient coherence feels like a property of the team rather than a property of its size. The trap is that this coherence is exactly what breaks at the next hiring round, when the context that lived in the founders' heads stops reaching the people doing the work. The concrete move is to write down the things that currently feel too obvious to write: the specs, the decision logs, the conventions that a new hire would otherwise have to absorb by osmosis. Not because anyone is lost today, but because Chapter 3's argument applies to teams as well as repositories — tacit standards that only live in senior heads are standards an organization cannot operationalize.

For a scale-stage company of fifty to two hundred people, the dominant failure mode is silo proliferation: every team builds its own AI tooling, every tool grows its own context, and none of that context is shared. The Bloomberg engineering organization, describing its deployment across a large specialized workforce, named the hard part directly — it was not model quality but scale itself. The productivity that showed up in greenfield work dropped off against hundreds of millions of lines of existing code, and getting past the early adopters meant treating rollout as an organizational program rather than a tooling decision: AI coding was folded into the existing onboarding program so that each new hire became, in their words, a change agent who carried the new way of working back to a more senior team. The lever at this scale is a shared layer through which AI capability is provided, measured, and governed once, instead of being reinvented locally a dozen times.

For an enterprise of hundreds or thousands of engineers, often regulated and often carrying decades of legacy, the prerequisite that the smaller-company conversation omits is legal and compliance as a design input rather than a later constraint. An enterprise cannot delegate agent access without knowing what the agent can see, what it can do, and who approved both. The pilot that lives in a sandbox indefinitely is usually not waiting for a better model. It is waiting for someone to design the credential scope and the audit trail that would let it move to production — which is the governance argument made concrete, and the subject of the rest of this chapter.

What cuts across all three scales is that the jumps are governance events, not capability events. A team does not become AI-native because the models got better. It becomes AI-native because someone made a structural decision — hardened a path, wrote down a convention, built a review system. The capability was usually available long before the organization could use it coherently. What unlocks the next stage is always an organizational act.

## Broader creation, narrower paths to ship

If execution is cheap, the natural move is to let more people execute — and the sharpest version of that move comes from Lisa Orr at Zapier, who states it as a deliberate provocation: "at Zapier we are empowering our support team to ship code." Not file tickets for engineers to ship code — ship it themselves. When the cost of a competent first implementation collapses, the historical reason for routing all code through a narrow guild of engineers weakens with it.

But the provocation only works with its other half. Broader creation does not mean a free-for-all; it means widening who can *start* work while narrowing and hardening the path by which work *ships*. The support engineer can open the pull request. What protects the company is that the path from that pull request to production runs through the same tests, the same evals, the same review gates, the same permission boundaries that any change runs through. Democratized creation and stronger governance are not opposites. They are the two things that have to rise together, or the first one becomes a liability.

This is why roles are blurring and new ones are appearing at the same time. James Lowe's argument that every product needs an AI product manager — and that it should be you — is really an argument that someone has to own the judgment layer that cheap creation now demands: deciding which of the many possible artifacts is worth shipping, and shaping the constraints under which non-specialists create safely. The practical move is to name that owner explicitly, because the judgment does not allocate itself once creation is cheap — the artifacts pile up and someone is implicitly deciding by default, usually whoever merges. Denys Linkov's work on structuring a modern AI team points the same direction with a concrete caveat: the capabilities that ship dependable AI — product sense, evaluation, infrastructure, domain knowledge — used to live in separate departments and now have to coexist on one team, but, as he puts it, they "don't necessarily need to be one person; they can be multiple people." The unit to design is the team's capability mix, not a single heroic hire.

The pressure reaches compensation and hiring too. Once output stops being a clean proxy for contribution, the incentive to pay for effort starts to misfire — Arman Hezarkhani's proposal to pay engineers "like salespeople," on outcomes rather than effort, is one early attempt to re-anchor reward to value in a world where effort is cheap. And hiring inherits a problem Beth Glenfield names directly — AI is "breaking how we hire technically": when everyone interviews with AI, the old signals of competence stop discriminating, and the organization has to learn to hire for judgment and taste rather than for the ability to produce code that an agent can now produce for anyone.

## Review becomes the bottleneck

When one person can direct many agents and more people can now create, the volume of work produced rises far faster than the human capacity to review it. Review — not generation — becomes the binding constraint, and the symptom is concrete: a review queue whose depth grows week over week while merge rate stays flat. Zack Proser, running parallel agents on WorkOS's Applied AI team, states the inversion plainly: "agents are not the bottleneck now and I think that's going to increasingly be the case, but we are." His point is that once you can hand an agent verification criteria and the tools to meet them, it will loop until it does — generation effectively scales without limit — while the human attention that has to confirm the result "still degrades under load. It's still the hard constraint." That is the structural fact behind the Monday-morning scene. The weekend produced a pile of pull requests, drafts, and automations; every one needs a human judgment somewhere before the organization can trust it; and the supply of trustworthy human judgment did not increase over the weekend. Maggie Appleton, describing collaborative AI engineering from inside GitHub, captures the pathology precisely — going fast without good alignment leads to wasted work, duplicate effort, and giant review queues with little context. The queue is where ungoverned speed goes to die.

The instinct to handle this by asking humans to review harder does not scale, because it asks the scarce resource to absorb the growth of the abundant one. The organizations that cope build the review function the way Chapter 4 argued they should build evals: as a system, not a heroic individual act. Layered validation, where automated checks and evals clear the routine cases so human attention concentrates on the consequential ones. Triage rules that decide what a human must see and what can ship on green. Roll-up visibility of the kind Eric Zakariasson describes in Cursor's software-factory work — a single surface that shows what every agent is doing and, crucially, what the human actually needs to look at — rather than a firehose of individual agent chatter.

The move that makes that automated tier affordable is worth naming on its own, because the obvious objection is that an automated check is only as trustworthy as the model behind it — and trusting a verdict you did not produce is the whole problem. The corpus's answer is that a more trustworthy verdict does not require a more expensive model; it requires redundancy. Because generation is cheap, you can buy reliability by sampling and aggregating rather than by upgrading. Aakanksha Chowdhery describes the base move — have "the models ... generate multiple responses and then do majority voting" — so an idiosyncratic error gets outvoted. Stronger versions diversify the voters: Alberto Romero runs a "multimodel consensus" across different model families with confidence-weighted voting, which cancels a single family's bias rather than just its variance, and a multi-agent literature review uses "multi-agent debate to get reliable state evaluation instead of" a single prompt. Leonard Tang at Haize Labs takes it furthest, having "weaker LLMs debate each other about what the stronger model is saying" to build judging systems that beat a frontier model at a fraction of the cost. The organizational lesson is that the automated tier of review is not a cheaper, worse stand-in for human judgment: built as a panel of diverse cheap models that vote or debate, it can be both cheaper and more trustworthy than one expensive call — and, just as usefully, it can surface its own low-confidence cases as exactly the queue a human should look at. The redundancy only pays off when the voters are genuinely independent; a panel of one model family sampled three times cancels noise but not shared bias.

The connection back to Chapter 4 is direct and load-bearing: in an AI-native organization, eval and review capacity is not a quality-assurance afterthought. It is the throughput limit of the entire company. You can only safely create as fast as you can trustworthily review.

## Alignment debt is the new invisible tax

There is a subtler failure than an overflowing review queue, and it is the one most likely to catch good teams by surprise. When individuals each direct their own fleet of agents in private, every workflow can be locally efficient and the whole can still be globally incoherent. Two engineers solve the same problem two different ways. A feature gets built that quietly conflicts with another team's assumptions. Work is duplicated, contradicted, or wasted — not because anyone was careless, but because alignment never happened.

Maggie Appleton names the root cause with unusual precision: "None of our current tools give teams a shared space to discuss plans, gather the right context, and work with agents as a collective." The tooling optimized the individual loop — one developer, many agents — and left the collective loop unbuilt. Each person's context lives in their own session. The plans never meet until the pull requests collide.

This is worth treating as a distinct organizational liability, and "alignment debt" is a useful name for it. Like technical debt, it accrues invisibly while things feel fast, and it comes due all at once — as the duplicated work, the conflicting implementations, the surprise feature nobody coordinated, the giant unmergeable pile. And like technical debt, the cure is not to slow down but to pay alignment earlier: to move shared planning, context-gathering, and visible work decomposition *upstream* of the agent fan-out, so that two dozen agents are working from one understood plan rather than two dozen private ones. The operational rule is that the plan, not the pull request, is where two people's work should first meet. If the first time two engineers' agent work touches is at the merge — the point Appleton diagnoses, where the plans never meet until the pull requests collide — alignment was already skipped, and the only question left is how expensive the collision is.

The deeper point is that as execution fans out, alignment has to move in the opposite direction — it has to concentrate and move earlier. The cheaper it is to start work, the more expensive it becomes to have started the wrong work in parallel twenty times. Alignment debt is the tax an organization pays for treating a collective activity as a collection of private ones.

## Governance as the load-bearing wall

The standard corporate framing treats governance as the brake: the process that slows things down for the sake of safety and compliance, so that the AI-native dream becomes minimizing it — finding the path of least institutional resistance from idea to deployed output. That framing is what produces the company that ships fast and then reverts, recalls, or absorbs a consequence it never budgeted for. The inversion this chapter argues for is that in an AI-native organization governance is not overhead at all; it is the mechanism by which the organization can afford to be fast. The test of whether you hold the second view rather than the first is simple: ask whether your governance work is something you do *before* you grant an agent access or something you bolt on *after* a pilot embarrasses you.

The organizations that have built durable AI-native practices describe the same relationship. Joel Hron, CTO of Thomson Reuters — a company deploying AI across legal, tax, and risk workflows where, as he puts it, the risks of being wrong are not particularly acceptable — describes the shift that gives this book its title: the north star has moved from assistants that are merely helpful to systems asked to produce output and make judgments and decisions on behalf of users. Making that move in high-stakes work is not primarily a model-quality question; it is a trust question. And the book's argument is that a colleague is trusted with consequential work not because they can never be wrong but because there is a system around them — accountability, escalation, review — that bounds the damage when they are. Agents earn colleague status the same way. Governance is not the obstacle to trust; it is what trust is built from.

The practical consequence is that governance is not a Phase 3 problem to be solved after the agents are running. It is a Day 1 design question. An agent running with unbounded credentials and no audit trail cannot be granted access to the production system — not because the risk team said no, but because there is no basis on which to say yes. Scoped credentials, bounded authority, and comprehensive audit logs do not constrain the agent's capability. They are what allows the agent to be deployed at all. Chapter 7 made this argument for the technical security model; here it applies at the organizational level.

The organizations succeeding with enterprise AI at scale — across finance, legal, and regulated health contexts — consistently describe the same enabling sequence: define the governance architecture first, then expand the scope of what agents can do within it. The pilot that lives in a sandbox indefinitely is usually not waiting for a better model. It is waiting for someone to design the credential scope, the audit trail, and the escalation path that would allow it to move from demonstration to production. The governance design is the unlocking work.

The resulting principle is a reframe of the question every enterprise team should be asking. Instead of "what is the minimum governance we can get away with?", ask "what is the lightest governance that earns us the trust to go fast?" The answer is always less than the instinctive first draft — layers of approval designed for human-speed work — and always more than the proof-of-concept default of no governance at all. The right amount is whatever makes the next grant of access defensible to the person who has to sign off on it.

## The company becomes a harness for its own agents

An organization that handles cheap creation, the review bottleneck, alignment debt, and governance is doing, at the scale of a company, exactly what Chapter 3 said a good codebase does for a coding agent: it has taken the judgment that used to live tacitly in senior heads and the hallway conversation and made it explicit, versioned, and available to humans and agents alike. Shared standards instead of folklore. Written policies instead of "ask Sarah." Permission systems instead of trust by default. Review gates instead of hoping. A company, in other words, is a harness for its own agents — and AI-native advantage is the quality of that harness.

That is also why the advantage is so hard to copy. A competitor can buy the same models and the same seats tomorrow. What it cannot buy overnight is an operating model in which institutional judgment has been packaged into reusable constraints — broad paths to create, narrow paths to ship, review built as a system, alignment paid upfront, and standards legible to the agents doing the work. That is built, not purchased, and the building is the moat.

## What this means for what endures

The AI-native organization is not the one with the most enthusiastic prompting culture, and it is not the one with the highest activity dashboards. It is the one that learned to convert cheap generation into trusted throughput — which turns out to require the least glamorous things in the building: clear standards, real review, honest measurement, and alignment that happens before the work fans out rather than after it collides.

Notice what that list does not contain. It does not contain a particular model, a particular vendor, or a particular protocol. Every concrete technology in this book will be replaced; some of it already has been between the talks that anchor these chapters and the page you are reading. What persists is the shape of the problem: cheap execution makes judgment scarce, scarce judgment has to be organized, and organizing it is an engineering discipline applied to an institution rather than a codebase.

The technical question and the organizational question, in the end, turn out to be the same question asked at two scales: how do you build a system in which delegated work compounds instead of fragmenting? The final chapter asks what survives when the models, the tools, and the org charts have all turned over again — and the answer it reaches for is already visible here, in the parts of the AI-native organization that were never really about AI at all.

## What to do with this

- Audit your engineering dashboards for activity-vs-outcome confusion. If you are reporting PRs opened, commits, or lines generated, you are counting the abundant resource; replace or supplement those with outcome measures — rework rate, share of generated work that ships unreverted, and time spent in the review queue — because counting artifacts in a world where artifacts are cheap is counting the wrong thing.

- Treat review and eval capacity as the company's throughput limit, not a QA afterthought. Build the review function as a system rather than asking humans to review harder: add layered validation so automated checks and evals clear routine cases, triage rules that decide what a human must see versus what can ship on green, and a single roll-up surface (à la Cursor's software-factory work) showing what each agent is doing and what the human actually needs to look at.

- Build the automated review tier as a panel, not a single judge. Trustworthy verdicts come from redundancy, not a bigger model: sample and majority-vote, or run a consensus of *diverse* model families that vote or debate, so independent errors cancel and the system can flag its own low-confidence cases for a human. Keep the voters genuinely independent — one model sampled three times cancels noise but not its own bias.

- Move alignment upstream of the agent fan-out. Make the plan, not the pull request, the place two people's work first meets — shared planning, context-gathering, and visible work decomposition before the agents start — so two dozen agents work from one understood plan instead of colliding at merge. Alignment debt accrues invisibly while things feel fast and comes due all at once.

- Widen who can start work, then harden the single path by which it ships. Take Lisa Orr's provocation seriously — let non-engineers (e.g., support) open pull requests — but route every change through the same tests, evals, review gates, and permission boundaries, because democratized creation and stronger governance have to rise together or the first becomes a liability.

- Name an owner for the judgment layer. James Lowe's case for an AI product manager amounts to this: someone has to decide which of the many cheap artifacts is worth shipping and shape the constraints under which non-specialists create safely. The point is not the title but the assignment — leave it unnamed and the decision still gets made by default, usually by whoever merges, which is the worst place to locate it.

- Close the last human-only path rather than buying more seats. The holdout process — the review, handoff, or plan that still assumes human-speed work — is the constraint. Partial delegation makes individual people faster inside an unchanged workflow; redesigning the workflow requires finding and converting that holdout. Identify it and convert it, because more seats in an unchanged process does not change the process.

- Diagnose which queue is backing up before you prescribe. The common error is to throw throughput solutions — more models, more seats, faster generation — at a problem that is really about coherence or review. Before adding capacity, ask which scarce resource the bottleneck is rationing: if engineers are faster but nothing ships sooner, the constraint has already moved off generation, and more generation makes it worse, not better.

- Design governance architecture before expanding scope. Ask not "what is the minimum governance we can get away with?" but "what is the lightest governance that earns us the trust to go fast?" The pilot that lives in a sandbox indefinitely is usually not waiting for a better model — it is waiting for someone to define the credential scope, the audit trail, and the escalation path that would make production access defensible. Do that work first.

- Write it down before you scale it. For small teams, the coherence that makes your current AI usage feel effortless is largely a property of proximity, and proximity is exactly what the next hiring round dilutes. Write the specs, decision logs, and conventions while they still feel too obvious to write down — not because anyone needs them today, but because the next ten people will, and Chapter 3's lesson holds here: a standard that lives only in senior heads is one the organization cannot operationalize.

---

# Chapter 10 — What Endures

A book about a field moving this fast has an obvious problem: by the time you read it, some of its examples will be obsolete. Models named in these chapters have already been superseded. Frameworks have been renamed. Tools that anchored an argument have shipped a version that changes the details. This final chapter is about the part that does not expire — the operating model that survives the churn, and the reason it survives.

Because the churn is real but shallow. New models arrive, old frameworks get rebranded, and every layer of the stack takes its turn claiming to be the one that finally matters. Underneath that surface, the engineering pattern has been remarkably stable across every chapter of this book. What endures is not a model or a framework. It is a way of turning machine capability into dependable work.

## The argument the whole book was making

This book opened on the shift from assistant to delegate, and argued from the first page that reliability comes far less from model cleverness than from the scaffolding around the model. By now that claim has been paid out nine times over, once per chapter, so it is worth stating what only the closing chapter can: the nine instances were never nine separate lessons. They were one structural fact viewed from nine angles. Omar Khattab points at it directly when he describes systems he built — DSPy among them — that "fundamentally stayed the same over the years" from the days of text-davinci-002 up to o4-mini, even as the models underneath were swapped out entirely. The model was the part that turned over. The structure was the part that stayed, and the staying is what the rest of this chapter is about.

It is one continuous consequence, not nine. Delegation stretched the failure surface from a single response across a whole workflow — the move Joel Hron at Thomson Reuters named when the north star went "from helpfulness to productive" — and every chapter after Chapter 1 took up one stretch of that surface: taste as the scarce input (Chapter 2), the harness that encodes it (Chapter 3), evals as the control system (Chapter 4), context as infrastructure (Chapter 5), runtimes that carry work across time (Chapter 6), identity and bounded authority as the price of acting (Chapter 7), realtime making every failure audible at once (Chapter 8), and the organization revealed as the same object at the largest scale — a harness for its own agents (Chapter 9). Seen from the end, they are not a sequence of topics. They are a single failure surface, chapter by chapter, refusing to be closed by a better model.

What makes that more than a tidy summary is where the durable practice actually lives. Ryan Lopopolo at OpenAI put it in a sentence — "the important thing is not the code but the prompt and the guardrails that got you there" — and the practice that follows is concrete and model-independent: the breadcrumbs, ADRs, and persona-oriented documentation that teach an agent what a good job looks like outlast any model that reads them. That is the thing most likely to still be true after the specific scaffolding in these chapters has been replaced.

## Why the principles outlast the tools

It is worth being precise about *why* the pattern endures, rather than just asserting that it does, because the reason is what makes it trustworthy — and the reason is mechanical, not sentimental. A better model does not retire the scaffolding; it raises the cost of its absence.

Better models do not remove the need for scaffolding. They raise the stakes on it. A more capable model executes a badly framed task more confidently, produces more plausible wrong output, and fails faster and at larger scale when the surrounding system is weak. Every improvement in raw capability makes the harness, the evals, the context discipline, and the authority boundaries matter *more*, because the blast radius of an unscaffolded mistake grows with the capability of the thing making it. This is the quiet inversion at the heart of the book: the better the models get, the more the engineering around them decides the outcome.

Khattab's own phrase for the goal — "engineering AI systems that endure" — comes paired with a name for the failure that breaks it: premature optimization, which he defines precisely as hard-coding at a lower level of abstraction than you can justify. His test is memorable. If you want a square root, *say* "give me a square root"; don't write the bit-manipulation that happens to appease today's machine. The AI-native version is the same move: express intent at the level of "find the relevant precedent" or "draft the migration," not at the level of one model's prompt quirks, and only drop to the lower level once you have demonstrated the higher one is not good enough. Bind your system to a model's idiosyncrasies and you have hard-coded the bit shifts; build the abstraction above them and the model becomes a replaceable component inside a structure that holds its shape when you swap it. Dax Raad puts the provocation at its sharpest: "AI changes nothing." He means it literally about the parts that were always hard — getting a stranger to stop scrolling and try your product, the marketing and distribution work that "hasn't changed" across his whole career and that AI "really hasn't done much for." He is wrong on the surface and right underneath: the interfaces and economics changed enormously, but the discipline of turning capability into dependable systems did not change at all. It got more important.

## The durable pattern: constrained delegation

If the book reduces to one transferable idea, it is **constrained delegation** — handing real work to a system while holding the bounds inside which it is allowed to do that work. The phrase is worth naming at the end rather than the start because it is the place every chapter's argument converges: give the system a clear task, a prepared working environment, the right slice of context rather than all of it, a way to preserve state across the gaps, powers narrow enough to trust and revoke, and human judgment focused on the consequential edges instead of spread thin or pretending it can be removed. Read backward, each chapter is one clause of that sentence made load-bearing — the harness is the prepared environment, evals are how you know the constraint is holding, context architecture is the right slice, runtimes are the preserved state, security is the narrow power, the human control plane is the judgment at the edges. The synthesis is that they are not six disciplines to balance. They are one discipline with six surfaces, and a system is only as delegable as its weakest one.

None of those pieces depends on a particular vendor, protocol, or model generation. They are the stable structure; the tools are the parts you replace inside it. A team that internalizes constrained delegation can adopt next year's model without rethinking how it works, because the model was always meant to be the swappable part. Mario Zechner's account of building in a world of slop is, at bottom, a constrained-delegation story, and his rules are specific enough to copy: scope a task so the agent is "guaranteed to find all the things it needs" to do it, modularize the codebase so that scoping is even possible, and give the agent a function to evaluate how well it did the job. Then sort by stakes — let the agent "wipe" the boring, non-critical code, but for critical code "read every line," because once an agent has written both your code and your tests you can no longer trust either without looking. The discipline, not the model, is what lets you use powerful, unreliable generation without drowning in its output.

## What this asks of the reader

The practical consequence is a reallocation of attention. The instinct in a fast-moving field is to chase the tools — to treat staying current with the newest model and framework as the core of the work. That instinct is not wrong, but it is not where the durable advantage lives. "Trusted throughput" is not an abstraction once you name its binding constraint: the limit is review capacity, the rate at which a human can trust what came back, which is why practitioners describe every engineer becoming, in effect, a code reviewer. Cheap generation lifts the ceiling on production and leaves that constraint untouched, so the structure worth building is the one that raises it — layered evals that catch the consequential cases, roll-up visibility so a person sees what the fleet did, the spec discipline that makes output legible enough to check quickly. The teams that win the next decade will not be the ones that adopted the newest tools fastest. They will be the ones that built that structure — and could therefore absorb every new tool without chaos.

So the question this book leaves you with is not "which model should I use?" It is the question that survives every answer to that one: what does the system around the model have to be, before I can trust what comes out of it? That question doubles as a checklist you can run against any AI feature before shipping it — clear intent, a prepared environment, the right slice of context rather than all of it, preserved state across the gaps, authority narrow enough to revoke, measurement that tells the truth instead of counting artifacts, and human review placed at the consequential, irreversible steps rather than spread evenly. The last item is the one teams get wrong most reliably: judgment spread thin everywhere is the same as judgment nowhere, and the discipline is choosing the irreversible edges — the credential, the checkout, the merge to main — and watching those. A gap in any one of these is where the next failure will come from, which is why each chapter spent its length on a single item. The list does not change when the model does. It is the part that endures.

Which returns us to where the book started — the assistant-to-delegate shift — but with the question it raised now answerable. Everything since Chapter 1 has been one answer at rising scale to "what has to be true before that trust is earned?", and the answer the high-stakes builders converged on is not "maximize autonomy" but treat agency as a dial, as Joel Hron put it — turned up where the work is reversible and recoverable, turned down where a wrong move costs something you cannot get back. That is the synthesis the opening could only gesture at: trust was never a property of the model to be waited for; it was a property of the dial, and the dial is something you build. The models will keep getting better, and that calibration will keep mattering more, not less. The colleague was never going to be trustworthy because it was clever. It is trustworthy, if at all, because of the structure we build around it — and that structure, not the cleverness, is what endures.

## What to do with this

- Invest in the scaffolding, not the model: harnesses, evals, context discipline, and authority boundaries are where the durable advantage lives, because a more capable model executes a badly framed task more confidently and fails faster and at larger scale when the surrounding system is weak. The blast radius of an unscaffolded mistake grows with the capability of the thing making it — so the better models get, the more this work pays off.
- Build so the model is a replaceable component. Treat it as the swappable part inside a structure that holds its shape when you swap it, rather than bonding your system to a particular model's quirks. A team that internalizes this can adopt next year's model without rethinking how it works.
- Run constrained delegation as a design template for every AI feature: give the system a clear task, a prepared working environment, the right slice of context (not all of it), a way to preserve state across the gaps, and powers narrow enough to trust and revoke — keeping human judgment focused on the consequential edges rather than spread thin or removed entirely.
- Before shipping any AI feature, run the closing checklist as a pre-ship test: clear intent, prepared environment, the right context, preserved state, bounded authority, measurement that tells the truth, and human judgment at the edges that matter. A gap in any one of those is where the next failure will come from.
- When deciding where to spend attention in a fast-moving field, reallocate it from chasing tools toward building the structure that turns cheap generation into trusted throughput — the teams that win the next decade are the ones who can absorb every new tool without chaos, not the ones who adopt the newest tool fastest.