One Task,
Three Passes

Getting useful work out of an AI agent

You’ve been told this changes everything.
It hasn’t worked yet.

Either
you haven’t really started. Opened it, typed something, got a shrug, closed it.

Or
you did start, and it produced something confident and wrong that took longer to clean up than to have written yourself.

Both are the correct reaction. The method is missing — nobody hands you the method, they hand you a text box.

TARGET 0:17

[1] "Either — you haven't really started. You opened it, typed something, got a shrug, and closed it."

ON SCREEN: Two boxes — Either you never really started, Or it gave you something confident and wrong — then the line that both reactions are correct.

[2] "Or — you did start, and it produced something confident and wrong that took longer to clean up than writing it yourself." "Who's in the left box? Who's in the right?" — show of hands if the room is warm, 10 seconds, don't take answers.

[3] "Both are the correct reaction. The method is missing — nobody hands you the method, they hand you a text box." "So today I'm not going to tell you it's great. I'm going to take one small task, run it three times, and let you watch it get better."

Steps: 3 steps, click through.

NEXT: "So here's what this hour is, and what it isn't."

STAGE: do not open on the pain of supervising a busy agent — this room hasn't felt it yet.

This starts where the 101 stops

Assumed: you have run a task in plan mode and read the diff.

By the end you can:

Explain why quality depends on context and specification, not model choice
Locate yourself on the map and name your next step
Judge what comes back instead of trusting how polished it looks
Capture one thing that worked into a file your team can reuse

Fluency takes weeks of doing this on real work. A talk doesn’t buy it.

TARGET 0:51

"Anyone who tells you a talk makes you fluent is selling something."

ON SCREEN: The prerequisite, then four numbered objectives, then the honest limit on what a talk can buy you.

[1] Explain why quality depends on context and specification, not model choice.

[2] Locate yourself on the map and name your next step.

[3] Judge what comes back instead of trusting how polished it looks.

[4] Capture one thing that worked into a file your team can reuse.

Steps: 4 steps, click through.

NEXT: "Before we start — roughly where are you on this?"

STAGE: MUST slide — this is the contract. If the room hasn't seen the 101, say so out loud and point them at it afterward; don't back-fill here. Name the AI Fluency framework only if it helps someone place the deck — don't teach it, the level map is this talk's spine. 4D: all four, one per objective.

Most of this room is at 2 —
which is the normal place to be

1
No AI

2
Individual
ad-hoc use

3
Team-integrated
workflows

4
Semi-autonomous
+ guardrails

5
Fully
AI-native

Level 1 is behind you. Three passes over one task make Level 2 deliberate; the rest of the hour is 3, 4 and 5. A map is not a scorecard — it just tells you which direction is forward.

The thing that changed isn’t quality.
It’s the size of task it can finish.

~5 hours

of human work, on tasks a
frontier model finishes half the time
(interval: 2.8–12.1 hours)

Measured across 228 tasks.

That horizon doubled roughly every
7 months since 2019 — and every
3 months since 2024.

METR, Time Horizon 1.1, January 2026

TARGET 1:47

"The thing that changed isn't quality. It's the size of task the model can finish."

ON SCREEN: ~5 hours in large type, what the measure actually is, and both doubling rates.

"This is the length of task — measured in how long a human expert takes — that the model completes about half the time. Not a claim that it runs five hours unattended."

"The doubling time is about seven months over the full period, three months since 2024."

Steps: 1 step, click through.

NEXT: "So let's take one task and actually run it."

STAGE: the only "models got better" claim in the talk — chosen because it's about autonomy duration, which is what the rest of the talk is about. Caveats if asked: interval is wide (2.8–12.1h); human baselines measured for only 5 of 31 tasks. No SWE-bench here — see slide 18 for why. If pressed on the harness confound: METR holds the harness fixed across models, so the slope survives even though the level doesn't — which is exactly why the next twenty minutes are about the scaffold.

The task: add retry with backoff

Our API client hammers a flaky upstream service.
Add retry with backoff.

Deliberately boring. Every one of you has done a version of it.

Hold in your head what a good answer looks like.

Pass 0: it’s good. Better than
most of us write by hand.

// Methods safe to replay: the upstream ends in the same state whether
// the request lands once or twice. POST is absent on purpose.
export const IDEMPOTENT_METHODS = new Set(['GET','HEAD','OPTIONS','PUT','DELETE']);

// Full jitter (random(0, backoff) rather than backoff itself) keeps
// concurrent callers from re-converging on the same retry instant.
export function backoffDelay(attempt, { baseDelay, maxDelay }, random = Math.random) {
  const ceiling = Math.min(maxDelay, baseDelay * 2 ** attempt);
  return Math.round(random() * ceiling);
}

// Retry-After is either delta-seconds or an HTTP date. Returns ms, or null.
export function retryAfterMs(response, now = Date.now()) { /* ... */ }

real output, one sentence of prompt — plus 18 passing tests it wrote unasked

TARGET 3:12

[8 steps] Arrow through the real output at your own pace. Stop on the POST comment and on the jitter comment — those are the two that carry the beat. The full source reveals after the last step.

ON SCREEN: The replay of the real run steps through the real output line by line; the code it produced reveals after the last step.

"This is genuinely good. Better than most of us write by hand. Keep watching."

Steps: 9 steps, click through.

NEXT: "So — nothing to fix. Which is exactly the problem."

STAGE: sell it straight, no wink — the whole beat depends on them believing it's good, because it is. POST excluded with a comment, full jitter explained, Retry-After honored and capped, timeout vs. caller aborts distinguished, 18 passing tests unasked. Replay source: assets/ai101/capture/s15-pass0.ansi (same real run as assets/example/capture/pass0.ansi — see assets/example/README.md). Sources: 00-pass0-client.js, 00-pass0-retry.js.

So what decision did it make for you?

// The timeout is per attempt, so each retry gets a full budget.

You passed timeout: 10_000. Three attempts plus backoff is ~30 seconds. There is no total budget anywhere.

It didn’t get this wrong. It decided it, wrote a comment justifying it, and moved on.

Nobody flags that line in review. The comment makes it look considered.

TARGET 4:15

"What decision did it make for you?" — not "what's the bug." Take two answers, 90 seconds, hard stop. Then reveal:

ON SCREEN: The question, then the comment it wrote, the arithmetic underneath it, and the punchline.

[1] "There it is. 'The timeout is per attempt, so each retry gets a full budget.'"

[2] "You passed a ten-second timeout. Three attempts plus backoff is about thirty seconds. There is no total budget anywhere." "It didn't get this wrong. It decided it, wrote a comment justifying it, and moved on."

[3] "Nobody flags that line in review. The comment makes it look considered."

If nobody got it: "Right — and that's the finding. It's not that you missed a bug. It's that there wasn't one to miss, and there was still a decision in there you never made."

"Let's take one more and move."

Steps: 3 steps, click through.

NEXT: "And that is not a bug you can prompt your way out of."

STAGE: verified by probe — 3 attempts, 601ms of backoff alone at worst-case jitter, on top of three full per-attempt timeouts. Lands directly on slide 18's evidence.

Instructions are intentions —
every gap you leave, it fills

It’s probabilistic. When you write a rule, you are shifting a probability — not setting a flag.

Not a defect you can prompt your way out of. It’s the shape of the thing.

The loop, the window, and why it has never seen your code — all in the 101.

Pass 1 changes one thing:
write the intent down first

Same task. Same model. Same tool.

Before I open a session, I write down what I actually want — in a file.

Not a formal document. Eight lines.

Eight lines, four minutes — every one
a decision it made for me last time

# Retry with backoff — intent

- Retry only idempotent requests: GET, HEAD, PUT, DELETE. Never POST.
- Retry only on 429, 502, 503, 504, and connection/timeout errors. Never 4xx.
- Exponential backoff with full jitter. Base 100ms, cap 5s.
- Respect an overall deadline passed by the caller; never exceed it.
- Max 4 attempts total, including the first.
- Log every retry with attempt number and reason.

Same model, same tool, different output

const RETRYABLE_STATUS = new Set([429, 502, 503, 504]);
const IDEMPOTENT = new Set(['GET', 'HEAD', 'PUT', 'DELETE']);

function backoffWithJitter(attempt) {
  const ceiling = Math.min(MAX_DELAY_MS, BASE_DELAY_MS * 2 ** attempt);
  return Math.random() * ceiling;            // full jitter
}

if (!IDEMPOTENT.has(method)) {
  return fetch(url, { method, ...options }); // never retry non-idempotent
}

excerpt — full source in the repo

What’s still missing from my spec?

What happens when retries are exhausted — throw, or degrade?

Does the deadline include the backoff sleeps?

Does any of this apply to streaming responses?

The specification is the bottleneck, not the model

The file outlives the session —
and context degrades before it fills

Across 18 frontier models, accuracy grows increasingly unreliable as input length grows.

And models do worse when the long context is logically coherent than when it’s shuffled.

More context is not free.

So you don’t hold state in a conversation.

You write it to disk and hand it over.

The file is the handoff.

Chroma Research, “Context Rot,” 2025

TARGET 9:18

[1] "Across eighteen frontier models, accuracy grows increasingly unreliable as input length grows."

ON SCREEN: The degradation finding on the left, the file-is-the-handoff callout on the right.

[2] "And models do worse when the long context is logically coherent than when it's shuffled."

[3] "More context is not free."

[4] "So you don't hold state in a conversation. You write it to disk and hand it over. The file is the handoff." "This works with any tool, including none of mine. It's the most transferable thing in this talk."

Steps: 4 steps, click through.

NEXT: "So for pass two, I open a second session."

STAGE: CUT-FIRST if running long, but you lose the "why files" justification. Do not name a specific degradation percentage — the source is qualitative, not a number.

Pass 2: a second session that never
watched the first one work

When the work is done, open a fresh session and ask it to review the result.

With the spec. Without the conversation that produced the code.

Not the same session. The reviewer must not have watched the author work.

Both serious findings were in the one
requirement I’d put in bold

Eight findings. Six were minor — a missing log line, an unreachable constant, a case-sensitivity nit. Both of the serious ones were in the deadline logic.

HIGH 1 — the deadline check reserves no room for the next attempt, and destroys the last usable response. One of four attempts burns on a 12ms window that could never succeed.

HIGH 2 — only undefined is guarded. deadlineMs: null fails every request with zero fetch calls; NaN slips through and silently disables the deadline entirely.

attempt 1/4 failed (HTTP 503); retrying in 100ms
THREW Error: deadline of 130ms exceeded after attempt 2 (timeout after 12ms)

TARGET 10:26

[1] "Eight findings. Six were minor — a missing log line, an unreachable constant, a case-sensitivity nit. Both of the serious ones were in the deadline logic — the one requirement I put in bold."

ON SCREEN: Eight findings summarised, the two serious ones, and the probe output reproducing one.

[2] "First: the deadline check reserves no room for the next attempt, and destroys the last usable response. One of four attempts burns on a twelve-millisecond window that could never succeed."

[3] "Second: only undefined is guarded. A null deadline fails every request with zero fetch calls; NaN slips through and silently disables the deadline entirely."

[4] "And here it is reproduced — not read off the page. Run."

Steps: 4 steps, click through.

NEXT: "Be careful about why this worked — it isn't what people assume."

STAGE: real findings from a real cold review — assets/example/03-review-findings.md. Say "both serious findings," never "every finding was in the deadline logic" — six of eight are elsewhere and minor. An earlier draft got this wrong by trusting the reviewer's own summary over its own detailed findings.

It works because it has your spec — not
because two agents are smarter than one

Revising its own reasoning with nothing external to check against doesn’t reliably improve — sometimes it gets worse.

Copies of a model debating each other produce consistency, not correctness.

Pass 2 works because the second session has something external to check against — your spec.

And it ran probes — every finding reproduced by executing code, not read off the page.

Huang et al., “LLMs Cannot Self-Correct Reasoning Yet,” ICLR 2024

TARGET 11:29

[1] A model revising its own reasoning with nothing external to check against doesn't reliably improve — and sometimes gets worse.

ON SCREEN: Three claims about self-correction, then what actually made the difference.

[2] Copies of a model debating each other produce consistency, not correctness. At equal compute, that loses to just sampling the model a few times and taking the majority.

[3] What makes pass two work is that the second session has something external to check against — your spec — and none of the author's investment in its own choices.

[4] And notice what the reviewer actually did: it ran probes. Every finding was reproduced by executing code, not read off the page. That's external feedback in the sense that matters.

Steps: 4 steps, click through.

NEXT: "Which brings me to the uncomfortable part of this talk."

STAGE: MUST slide — the most intellectually honest beat in the talk. Do NOT say "a second agent never helps" — the paper's own taxonomy counts other models as external feedback too. It supports executable gates, not stacking symmetric LLM reviewers; don't overclaim it, someone in the room may have read it. Huang et al., ICLR 2024.

You cannot tell how well this is going.
Neither can I.

Polish suppresses scrutiny. Across 9,830 conversations, the ones involving code were where people least questioned the reasoning (−3.1pp) or noticed missing context (−5.2pp). Anthropic AI Fluency Index, Feb 2026

The harness moves the number as much as the model. Swap the scaffold around a fixed model and benchmark scores move like a model generation — 21.8 points on one benchmark. Gorinova et al., June 2026

And this one happened to me. I asked an agent to check a 2026 result. It reported +18% — twice. The paper reports −18%, a CI crossing zero, and “very weak evidence.” METR, Feb 2026

The check has to be structural, not a feeling

TARGET 12:03

[1] "Polish suppresses scrutiny. Across almost ten thousand real conversations, the ones involving code and artifacts were the ones where people were LEAST likely to question the reasoning, or notice something was missing."

ON SCREEN: Three pieces of evidence, each with its source, then the key message.

[2] "The harness moves the number as much as the model does. Swap the scaffold around a fixed model, and coding-benchmark scores move by a margin comparable to an entire model generation — a 21.8-point swing, on one benchmark."

[3] "And this one happened to me, while I was building this talk. I asked an agent to check a 2026 result. It told me plus eighteen percent — twice. The actual paper says minus eighteen, a confidence interval that crosses zero, and the authors call it 'very weak evidence.'"

[4 / key-message] "The check has to be structural. Not a feeling."

Steps: 4 steps, click through.

NEXT: "So that's level two — and you did all of it by hand."

STAGE: hold half a beat on [3] before advancing — it's first-hand, let it land. Say "while I was building this talk" — that's what makes it yours, not a stat. Do not say "everyone quotes it wrong" (unverified); do not credit the 200k-run figure to Gorinova (it's a study they cite, not their own). 4D: Discernment.

Level 1 → 2 — done

You just did the whole thing by hand

1
No AI

2
Individual
ad-hoc use

3
Team-integrated
workflows

4
Semi-autonomous
+ guardrails

5
Fully
AI-native

Write the intent to a file before you start. Hand the file to a fresh session. Open a second session to check the result against the file.

That is Level 2, and it is genuinely most of the value. If you stop here you have not failed.

Level 2 → 3

Your practice lives in your head.
That doesn’t scale to a team.

1
No AI

2
Individual
ad-hoc use

3
Team-integrated
workflows

4
Semi-autonomous
+ guardrails

5
Fully
AI-native

You know to say “never retry a POST.” The next person doesn’t. Every session on your repo starts from zero.

The highest-leverage file in the repo
is the one the agent reads first

# AGENTS.md

## Things an agent would otherwise get wrong

- Never retry non-idempotent requests. POST is not safe to replay here;
  the orders endpoint is not keyed for deduplication.
- `pricing.internal` is behind a 200ms p99. Anything in the request path
  needs a deadline, not just a per-attempt timeout.
- Run `npm test` before claiming done. There is no CI on this repo yet.

Not documentation. The set of things that get got wrong.

Ours is one line long

@AGENTS.md

That is the entire CLAUDE.md. The tool-specific file is a pointer; the content lives in a tool-agnostic one.

In May, ours was a “comprehensive project reference.” We learned.

Most of these files make things worse

A February 2026 study ran four agents across 300 SWE-bench Lite tasks and a 138-task agent benchmark, with and without context files.

Auto-generated context files — the raw output of an init command — did not improve success rates at all, while raising cost by over 20%.

And when they stripped the README and docs out of the repo, context files started helping consistently — which tells you most of what people put in them was already discoverable.

ETH Zurich, February 2026

TARGET 14:53

[1] "A February 2026 study ran four agents across three hundred SWE-bench Lite tasks and a 138-task agent benchmark, with and without context files."

ON SCREEN: The study, the null result on success rate, the cost, and the ablation.

[2] "The default advice everywhere is run init and commit the file it makes. The measurement says that move buys nothing, and costs over twenty percent."

[3] "And when they stripped the README and docs out of the repo, context files started helping consistently — which tells you most of what people put in them was already discoverable."

Steps: 3 steps, click through.

NEXT: "There's a budget here, and it's smaller than you think."

STAGE: be precise on effect size — the cost figure is solid, the success-rate deltas are small (~0.5–2pp) and not significance-tested; don't say "lowered success rates" as a clean negative. If pressed: two of five authors are from a commercial lab, and the 138-task benchmark is this same paper's own.

You have an instruction budget,
and it is smaller than you think

Instruction-following degrades as instruction count rises — benchmarks put the turning point somewhere around the low hundreds, and your file is not the first claim on that budget. The harness ships its own.

And adherence decays through a session: strong in the first couple of exchanges, measurably weaker by the tenth.

You’re not writing laws. You’re writing suggestions.

Every line you add is a line
that competes with the work

Bloated

“This is a Node.js project using ES modules. We value clean code and readability. Please write good tests. Follow SOLID principles…”

True, generic, and it displaces context that isn’t.

Sharp

“There is no CI on this repo. Run npm test before claiming done.”

Unguessable, specific, changes behaviour.

Test for every line: could a competent stranger have guessed it?

TARGET 16:18

[1] "Bloated: 'This is a Node.js project using ES modules. We value clean code and readability. Please write good tests...' All true, all generic — and it displaces context that isn't."

ON SCREEN: A bloated file and a sharp one side by side, then the one test that separates them.

[2] "Sharp: 'There is no CI on this repo. Run npm test before claiming done.' Unguessable, specific, changes behaviour."

[3] "Test for every line: could a competent stranger have guessed it? If yes, cut it."

Steps: 3 steps, click through.

NEXT: "And it isn't one file — context arrives in layers."

STAGE: the generic version isn't harmless — the window is finite, and everything you load is space the real work doesn't get. This one test fixes most bloated AGENTS.md files.

Context arrives in layers,
and you are budgeting, not tidying

Global — how you work, everywhere

Project — how this repo works (AGENTS.md)

Skills — loaded only when relevant

Tools / MCP — systems it can reach

Conversation — right now

The mistake is stuffing everything into the top layer.

TARGET 16:52

[1] "Global — how you work, everywhere."

ON SCREEN: Five layers, one at a time, then the mistake almost everyone makes.

[2] "Project — how this repo works. That's AGENTS.md."

[3] "Skills — loaded only when relevant."

[4] "Tools and MCP — systems it can reach."

[5] "Conversation — right now."

[6] "The mistake is stuffing everything into the top layer." "When I'm asking what museums to visit in Venice, it doesn't need to know I hate mushrooms. Both are true about me — only one is relevant, and the other one costs."

Steps: 6 steps, click through.

NEXT: "The third layer is the one worth understanding properly."

STAGE: CUT-FIRST — covered by the slide before and the lazy-loading slide after; move the Venice line up one slide if cut. Callback to slide 14 — layering is budgeting, not tidying.

A skill is context that loads itself
only when it’s needed

---
name: incident-response
description: Use when triaging a production alert, paging, or writing a
  postmortem. Covers severity levels, comms templates, and escalation.
---

# Incident response
(...900 lines the agent only reads when the description matches...)

The description is the whole game. It is the only part always loaded.

Precision in, precision out

	Before	After
Session cost	$3.60	$2.64
Wall time	17m 10s	9m 12s
New tokens	~361k	~169k
Lines the agent changed	1,159	119

Same task, same repo. The only change was tightening skill descriptions and deferring bodies behind them. No rules were deleted.

Three ways to give it capability,
and they are not interchangeable

Command / SOP	A repeated process	“run our design phase”
Skill	Domain knowledge	“how we handle incidents”
MCP tool	An action in another system	“query the dashboard”

Most teams reach for the third when they needed the first.

MCP is the expensive one,
and it charges you up front

MCP tool definitions load eagerly — every tool, every session, whether you use it or not. One internal server measured at ~27k tokens, about 13.5% of the window, before any work started.

A skill costs you a name and one line until it fires.

So: MCP for what you need in most tasks. Skills for what you need sometimes.

Which one would you reach for?

“Every new service needs the same six setup steps”

“The agent should be able to read our on-call schedule”

“Nobody remembers our deprecation policy”

Command. MCP tool. Skill.

Level 2 → 3 — done

The team’s knowledge stops being
something people carry

What you were doing by hand: re-supplying the conventions, every session, forever.

What replaced you: AGENTS.md for the corrections that repeat, skills for knowledge that loads on demand, MCP only where the agent must reach another system.

Do this week: at the end of your next session, ask what you had to correct, and put those lines in AGENTS.md.

Level 3 → 4

You are still the first person
who reads everything

1
No AI

2
Individual
ad-hoc use

3
Team-integrated
workflows

4
Semi-autonomous
+ guardrails

5
Fully
AI-native

The work got faster. Your queue didn’t. Everything still routes through one pair of eyes — yours.

Things you ask for, and things
you actually enforce

Asked for

“Always run the tests before you say you’re done.”

A probability shift. Usually works. Not always.

Enforced

A hook that runs the tests on every edit, and blocks on failure.

A door that is locked.

Autonomy is only as safe as the things you enforce

TARGET 22:23

[1] "Asked for: 'always run the tests before you say you're done.' That's a probability shift. Usually works. Not always."

ON SCREEN: Asked-for and enforced side by side, then the key message.

[2] "Enforced: a hook that runs the tests on every edit, and blocks on failure. That's a door that's locked."

[3] "Autonomy is only as safe as the things you enforce."

Steps: 3 steps, click through.

NEXT: "And someone measured exactly this, properly."

STAGE: direct callback to slide 9 — instructions are best intentions, not constraints, turned into a design principle. You can only safely stop watching what you've made impossible; everything else you're still trusting. Hooks fire on lifecycle events, before a tool runs, after an edit — they're code, so they always run.

Someone measured this properly:
five approaches, 600 runs each

	Pass rate	Input tokens
No instructions	15.7%	1,870
Simple instructions	82.5%	2,329
Full procedure in the prompt	99.8%	9,879
Graph / workflow orchestration	80.8%	3,116
Enforced by hooks	100.0%	3,346

Same tools, one task. Enforcement matched the best prompt at a third of the tokens — a hook fires at the decision, not ten turns earlier.

Clare Liguori, Strands Agents, March 2026 — one chatbot task, vendor-published

TARGET 22:57

"Six hundred runs per row. Writing the full procedure into the prompt does work — 99.8% — so the instinct isn't wrong, just expensive, and it degrades."

ON SCREEN: Five approaches, pass rate and token cost each, hooks on the bottom row.

"The hook wins because of WHEN it runs — at the decision point, reading what actually happened, instead of hoping a rule from ten turns ago survived."

Steps: 1 step, click through.

NEXT: "So you put the reviewers in front of your eyes, not after them."

STAGE: strongest empirical slide in the talk. Do not hide the workflow row — graph orchestration scored 80.8%, worse than a plain prompt, and slide 42 recommends a graph; the honest reconciliation is that a graph is brittle exactly where the task leaves the planned path, which argues for specifying first, not against graphs. State the limits unprompted: one chatbot task, vendor-published, model unnamed — don't say "same model." If asked: prototype with prose, convert to enforcement once the workflow stabilizes; enforcement only works on closed sets.

So put the reviewers before your eyes,
not after

Gate 1 — evaluate. Correctness, tests, security, style. Several reviewers, different jobs, in parallel.

Gate 2 — challenge. An adversary tries to refute gate 1’s findings. Are these real, or noise?

Gate 3 — safety. What did everyone else miss? Edge cases, error paths, concurrency.

By the time it reaches you, the question is “is this the right approach?” — not “is there a null check missing on line 47?”

TARGET 23:48

[1] "Gate one — evaluate. Correctness, tests, security, style. Several reviewers, different jobs, in parallel."

ON SCREEN: Three gates — evaluate, challenge, safety — then what's left for you to judge.

[2] "Gate two — challenge. An adversary tries to refute gate one's findings. Are these real, or are they noise?"

[3] "Gate three — safety. What did everyone else miss? Edge cases, error paths, concurrency."

[4] "By the time it reaches you, the question is 'is this the right approach' — not 'is there a null check missing on line forty-seven.'"

Steps: 4 steps, click through.

NEXT: "Gate one has a trick to it that's worth saying out loud."

STAGE: Gate 2 is the non-obvious one, worth dwelling on — a pile of reviewers produces a pile of findings, and most of the cost of a bad pipeline is false positives. Something has to argue the other side; gate 2's job is to kill findings, not add them.

The reviewers are told to forget
that you meant well

Reviewers run cold — the instruction to respect the author’s intent is suspended. They judge the code, not the plan.

Then a separate pass puts intent back and sorts every finding into real bug, over-flagged, or not a bug.

Two passes, because doing both jobs at once produces a reviewer that talks itself out of real findings.

Every loop needs a way to stop
that isn’t you noticing

A round cap. Three passes, then it stops and escalates rather than grinding.

A circuit breaker. If the same finding survives two attempted fixes, stop trying and hand it back.

A diminishing-returns breaker. If a round is producing less than the one before it, that is the signal to stop.

Without these, a loop either runs forever or quietly ships something it couldn’t fix.

TARGET 24:56

[1] "A round cap. Three passes, then it stops and escalates rather than grinding."

ON SCREEN: Three stop conditions, one at a time, then what happens without them.

[2] "A circuit breaker. If the same finding survives two attempted fixes, stop trying and hand it back."

[3] "A diminishing-returns breaker. If a round is producing less than the one before it, that's the signal to stop."

[4] "Without these, a loop either runs forever, or quietly ships something it couldn't fix."

Steps: 4 steps, click through.

NEXT: "Because the bill for missing these is real, and it's quiet."

STAGE: each exists because the absence of it broke something — say that, it's what separates this from a feature list. The circuit breaker is the interesting one: after two failed attempts, more attempts don't help — the model isn't converging, it's resampling.

The bill for getting this wrong
arrives quietly

Publicly reported: a runaway agent loop that burned tens of thousands of dollars in five hours. Another that ran eleven days before anyone noticed.

The mechanism is simple. Every retry re-reads the whole context. Iteration one costs a little. Iteration ten costs a lot. You pay for each failure again and again.

Agents don’t fail loudly. They retry, reformulate, and look busy the whole time.

If you can’t grep why it stopped,
you haven’t automated it — you’ve hidden it

DROP     gate=1 id=f-4a2 reason=refuted-by-adversary
DEMOTE   gate=2 id=f-7c1 reason=style-not-correctness
ESCALATE gate=3 id=f-9e0 reason=concurrency-merge-block
BREAKER  gate=1 id=f-4a2 reason=two-failed-revisions

Every decision the pipeline makes leaves a line. Including the decisions to not do something.

You can only let it run unattended
if the worst case is cheap

git worktree add ../task-3 -b task-3
cd ../task-3
# agent works here. If it destroys everything:
git worktree remove ../task-3 --force

Blast radius: exactly one task. Every mistake becomes a two-way door.

Isolated and unambiguous tasks
can run at the same time

step-1 ──┬──> step-3 ──> step-4
step-2 ──┘

The plan is a graph, validated by topological sort so nothing can depend on something that never runs. Code computes which tasks are ready, dispatches those, and recomputes when one finishes.

Note the order: specify, then isolate, then parallelise. Skip either of the first two and you get N confidently different answers.

Which of these could you safely
stop watching tomorrow?

“It always runs the tests”

“It never touches files outside the task”

“It writes good commit messages”

“It never force-pushes to main”

The ones you can enforce. Three of these are a hook or a worktree. One is a hope.

TARGET 28:12

"Which of these could you safely stop watching tomorrow?" — "it always runs the tests," "it never touches files outside the task," "it writes good commit messages," "it never force-pushes to main."

ON SCREEN: Four claims; which of them are enforceable reveals after you've taken answers.

[fragment] "The ones you can enforce. Three of these are a hook or a worktree. One is a hope."

"Your autonomy ceiling is set by what you can enforce, not by how much you trust it."

Steps: 1 step, click through.

NEXT: "That's level four."

STAGE: retrieval beat, 90 seconds, hard stop. Tests (hook), file scope (worktree), force-push (hook or branch protection) are all enforceable; commit message quality is judgment, you can ask but not guarantee. Q&A BACKUP: Clare Liguori — same author as slide 35's hooks data — published this same claim independently, Sep 2026. If asked "does anyone else say this," this is the answer.

Level 3 → 4 — done

You stop being the first reader

What you were doing by hand: being the only quality gate, on everything, serially.

What replaced you: layered gates that run before you look, bounded so they stop, logged so you can audit, inside isolation so the worst case is recoverable.

Do this week: add one hook that enforces something you currently ask for.

Level 4 → 5

And here is where I stop
being able to sell you anything

1
No AI

2
Individual
ad-hoc use

3
Team-integrated
workflows

4
Semi-autonomous
+ guardrails

5
Fully
AI-native

Level 5 means agents owning operational loops end to end. We are not there, and I’d be lying if I implied otherwise.

What actually defines each level
is who verifies the work

Levels 1 to 3 all rely on a human saying “this is good.”

Levels 4 and 5 delegate verification itself to the system.

The prerequisite for more autonomy isn’t a smarter model.
It’s better proof.

TARGET 30:36

"Levels one to three all rely on a human saying 'this is good.' Levels four and five delegate verification itself to the system."

ON SCREEN: Levels 1-3 versus 4-5 on who verifies, then the thesis of the whole talk.

"The prerequisite for more autonomy isn't a smarter model. It's better proof."

Steps: 2 steps, click through.

NEXT: "Here's that same idea as one concrete failure of ours."

STAGE: THE THESIS — the right place for it, after four levels the room can feel it's true. Reframes everything preceding as proof infrastructure: the spec, the cold reviewer, the gates, the hooks, the tests. Explains why level 5 isn't close — not weak models, but we can't yet prove correctness well enough to stop looking. Q&A BACKUP: same convergence as slide 43 — Clare Liguori reaches this from the guardrails side, independently, Sep 2026.

The instruction that says
“rate this minor” does not hold

We wrote a reviewer that was supposed to emit a specific severity. Across repeated runs on fixed inputs, it emitted whatever it wanted.

The fix was not a better sentence. It was moving the rule out of the prompt and into a deterministic check.

Prose is a hint. Code is a guarantee.

TARGET 31:27

"We wrote a reviewer that was supposed to emit a specific severity. Across repeated runs on fixed inputs, it emitted whatever it wanted."

ON SCREEN: The severity reviewer that ignored its own instruction, the fix, and the rule.

"The fix wasn't a better sentence. It was moving the rule out of the prompt and into a deterministic check."

"Prose is a hint. Code is a guarantee."

Steps: 2 steps, click through.

NEXT: "And here's one where we got it wrong in the other direction."

STAGE: don't quote a specific run count — small-n internally, the lesson is reproducible and that's enough. The single most transferable thing in the talk — whenever you catch yourself writing "the agent MUST," ask if a check can enforce it instead. Third callback to slide 9.

We built guardrails so good
they made everything worse

Review depth was scaled by how dangerous a change looked. Sensible — until you notice that editing a prompt file counted as touching the pipeline, so the cheapest, most reversible changes got the heaviest review.

A one-line prose edit, fully undone by git revert, went through the full production gauntlet.

It ran like that for weeks before anyone noticed. Nothing failed. It just quietly made every small change feel like shipping to production.

TARGET 32:01

"Review depth was scaled by how dangerous a change looked. Sensible — until editing a prompt file counted as touching the pipeline, and the cheapest, most reversible changes got the heaviest review."

ON SCREEN: How review depth was scaled, what it did to the cheapest changes, and how long it ran.

"A one-line prose edit, fully undone by git revert, went through the full production gauntlet — for weeks, before anyone noticed. Nothing failed. It just quietly made every small change feel like shipping to production."

Steps: 2 steps, click through.

NEXT: "Even the word autonomous is doing some work here."

STAGE: the most important, least flattering slide in this segment — tell it straight. The failure mode isn't that guardrails don't work, it's that they work in the wrong direction and nothing alerts you, because "more review" never looks like a bug. The fix was deletion: remove the escalation machinery, decide depth once by size, keep the adversary reviewers and the merge-block. Land: match rigor to stakes — same discipline as pass 1, you have to specify what "risky" means or something will decide for you.

And our “autonomous” mode
is less autonomous than it sounds

	Interactive	Auto
Who invokes it	you type the command	you type the command
Task sequencing	one file at a time	one file at a time
Context reset	you clear it	you clear it
Quality gates	you confirm	it decides and logs why

Every row is identical except one. What we automated was judgment at the gates — not the sequencing, and not the human.

What none of this does

It does not decide what to build.

It does not know which of these two designs you will regret in a year.

It does not know that this service is the one that pages you at 3am.

It does not know when the honest answer is “we shouldn’t.”

Every parallel task is executing your design. Good execution amplifies a bad one.

TARGET 33:26

Lead-in, already on screen: "It does not decide what to build."

ON SCREEN: Four things it cannot decide, one at a time, then the amplification warning.

[1] "It does not know which of these two designs you'll regret in a year."

[2] "It does not know that this service is the one that pages you at 3am."

[3] "It does not know when the honest answer is 'we shouldn't.'"

[4] "Every parallel task is executing your design. Good execution amplifies a bad one."

Steps: 4 steps, click through.

NEXT: "One last thing, about how this actually got built."

STAGE: the room's experience is the asset — pitch it that way, not as reassurance. Judgment about what to build and what not to build doesn't get automated, and the room already has it. Bad design doesn't get caught by good execution — good execution ships it faster, in more places, in parallel.

How this got built

None of it was designed up front

You have just seen four levels in twenty-five minutes, in a tidy order.

That is not how it happened, and if you copy the tidy order you will build the wrong things first.

Three months, twenty-seven projects,
and it ends with a deletion

Started by adopting an open-source workflow. Not building one.

Then the parts that broke: the handoff contract between phases, because context was getting lost.

Then gates, because output was landing that shouldn’t have.

Then isolation and parallelism, because by then it was safe to run more than one.

Then measurement — what are these gates costing versus what are they catching?

Then deletion. The most recent project removed machinery, and the system got better.

TARGET 34:34

Lead-in, already on screen: "Started by adopting an open-source workflow. Not building one."

ON SCREEN: Six steps in the order they actually happened, ending on a deletion.

[1] "Then the parts that broke — the handoff contract between phases, because context was getting lost."

[2] "Then gates, because output was landing that shouldn't have."

[3] "Then isolation and parallelism, because by then it was safe to run more than one."

[4] "Then measurement — what are these gates costing, versus what are they catching?"

[5] "Then deletion. The most recent project removed machinery, and the system got better."

Steps: 5 steps, click through.

NEXT: "You can watch it happen in a single file."

STAGE: real history, not a parable — first commit was in May, most recent project was a removal. Every layer was added in response to an observed failure, never an anticipated one. If you remember one thing: you are allowed to delete.

One file’s life, in lines

324 lines in May → 1,022 at its peak in July → 777 the following week.

The cut removed no rules. It removed machinery.

TARGET 35:08

"324 lines in May, 1,022 at its peak in July, 777 the following week. The cut removed no rules. It removed machinery."

ON SCREEN: One file growing and then being cut, with the three numbers under it.

Steps: 2 steps, click through.

NEXT: "And the core of this isn't ours."

STAGE: CUT-FIRST — the slide before already carries the build-then-delete arc in words. Two details worth saying out loud: the tier machinery removed had been added fifteen days earlier, and one piece of telemetry was added and deleted within twenty-four hours. Not embarrassing — the actual rhythm: you can't tell what earns its keep until it runs. About 4,600 net lines deleted that day; the process change (simplification as the default stance) landed BEFORE the deletions, not after.

We adopted the core.
We didn’t invent it.

Prompt-Driven Development — an open-source family of agent SOPs. strands-agents/agent-sop, Apache 2.0, published out of AWS’s open-source org, built by James Hood and a community of practitioners.

Summarise the codebase → hone the idea into a design → break the design into tasks → implement each task. The contracts between the phases are the valuable part — someone already worked out what a design document must contain for the next phase to consume it.

Our plugin wraps that with the gates and the graph. The wrapper is ours. The core isn’t, and you shouldn’t reinvent either.

TARGET 35:59

"Prompt-Driven Development — an open-source family of agent SOPs. strands-agents/agent-sop, Apache 2.0, out of AWS's open-source org, built by James Hood and a community of practitioners."

ON SCREEN: PDD — the repo, the licence, the four phases, and what part is actually ours.

"Summarize the codebase, hone the idea into a design, break the design into tasks, implement each task. The contracts between the phases are the valuable part — someone already worked out what a design document must contain for the next phase to consume it."

"Our plugin wraps that with the gates and the graph. The wrapper is ours. The core isn't, and you shouldn't reinvent either."

Steps: 2 steps, click through.

NEXT: "Yours starts far smaller than any of this."

STAGE: credit specifically — a talk about provenance can't be vague about where its own tools came from. The contracts point is the one worth stressing: months of work nobody should repeat, and it's free.

Yours starts as one markdown file

Next time you finish a session where you got something genuinely useful done, open a fresh session and say: “Look at what I asked for in the previous session. Capture what worked into a reusable instruction set. Save it as workflow.md.”

Every layer you have seen today started exactly there and got refined each time it was reused.

Three rules that survived every level

1. Clear context is your best friend.

2. Deterministic work belongs to code, not agents.

3. Plan for humans, not agents — every phase leaves a file you can read, and a door back in.

TARGET 37:07

[1] "One: clear context is your best friend."

ON SCREEN: Three rules, one at a time.

[2] "Two: deterministic work belongs to code, not agents."

[3] "Three: plan for humans, not agents — every phase leaves a file you can read, and a door back in."

"Everything I have shown you is one team's answer. The framework underneath it is somebody else's — it's better tested than mine, and it's free."

Steps: 3 steps, click through.

NEXT: "So — last question, and this one's for you."

STAGE: rule 3 keeps the rest honest — never REQUIRE you at every step, but always ACCOMMODATE you at any step. One closing sentence only: name Anthropic's four words (delegation, description, discernment, diligence — four of the 101's six chapter names) as what to go read. 4D: all four.

So — where are you, and
what’s the one step?

1
No AI

2
Individual
ad-hoc use

3
Team-integrated
workflows

4
Semi-autonomous
+ guardrails

5
Fully
AI-native

Write the spec down before you start

Put the corrections in AGENTS.md

Enforce one thing you currently ask for

Measure what a gate costs you

Be honestly sceptical

One Task,Three Passes

You’ve been told this changes everything.It hasn’t worked yet.

This starts where the 101 stops

Most of this room is at 2 —which is the normal place to be

The thing that changed isn’t quality.It’s the size of task it can finish.

The task: add retry with backoff

Pass 0: it’s good. Better thanmost of us write by hand.

So what decision did it make for you?

Instructions are intentions —every gap you leave, it fills

Pass 1 changes one thing:write the intent down first

Eight lines, four minutes — every onea decision it made for me last time

Same model, same tool, different output

What’s still missing from my spec?

The file outlives the session —and context degrades before it fills

Pass 2: a second session that neverwatched the first one work

Both serious findings were in the onerequirement I’d put in bold

It works because it has your spec — notbecause two agents are smarter than one

You cannot tell how well this is going.Neither can I.

You just did the whole thing by hand

Your practice lives in your head.That doesn’t scale to a team.

The highest-leverage file in the repois the one the agent reads first

Ours is one line long

Most of these files make things worse

You have an instruction budget,and it is smaller than you think

Every line you add is a linethat competes with the work

Context arrives in layers,and you are budgeting, not tidying

A skill is context that loads itselfonly when it’s needed

Precision in, precision out

Three ways to give it capability,and they are not interchangeable

MCP is the expensive one,and it charges you up front

Which one would you reach for?

The team’s knowledge stops beingsomething people carry

You are still the first personwho reads everything

Things you ask for, and thingsyou actually enforce

Someone measured this properly:five approaches, 600 runs each

So put the reviewers before your eyes,not after

The reviewers are told to forgetthat you meant well

Every loop needs a way to stopthat isn’t you noticing

The bill for getting this wrongarrives quietly

If you can’t grep why it stopped,you haven’t automated it — you’ve hidden it

You can only let it run unattendedif the worst case is cheap

Isolated and unambiguous taskscan run at the same time

Which of these could you safelystop watching tomorrow?

You stop being the first reader

And here is where I stopbeing able to sell you anything

What actually defines each levelis who verifies the work

The instruction that says“rate this minor” does not hold

We built guardrails so goodthey made everything worse

And our “autonomous” modeis less autonomous than it sounds

What none of this does

None of it was designed up front

Three months, twenty-seven projects,and it ends with a deletion

One file’s life, in lines

We adopted the core.We didn’t invent it.

Yours starts as one markdown file

Three rules that survived every level

So — where are you, andwhat’s the one step?

One Task,
Three Passes

You’ve been told this changes everything.
It hasn’t worked yet.

Most of this room is at 2 —
which is the normal place to be

The thing that changed isn’t quality.
It’s the size of task it can finish.

Pass 0: it’s good. Better than
most of us write by hand.

Instructions are intentions —
every gap you leave, it fills

Pass 1 changes one thing:
write the intent down first

Eight lines, four minutes — every one
a decision it made for me last time

The file outlives the session —
and context degrades before it fills

Pass 2: a second session that never
watched the first one work

Both serious findings were in the one
requirement I’d put in bold

It works because it has your spec — not
because two agents are smarter than one

You cannot tell how well this is going.
Neither can I.

Your practice lives in your head.
That doesn’t scale to a team.

The highest-leverage file in the repo
is the one the agent reads first

You have an instruction budget,
and it is smaller than you think

Every line you add is a line
that competes with the work

Context arrives in layers,
and you are budgeting, not tidying

A skill is context that loads itself
only when it’s needed

Three ways to give it capability,
and they are not interchangeable

MCP is the expensive one,
and it charges you up front

The team’s knowledge stops being
something people carry

You are still the first person
who reads everything

Things you ask for, and things
you actually enforce

Someone measured this properly:
five approaches, 600 runs each

So put the reviewers before your eyes,
not after

The reviewers are told to forget
that you meant well

Every loop needs a way to stop
that isn’t you noticing

The bill for getting this wrong
arrives quietly

If you can’t grep why it stopped,
you haven’t automated it — you’ve hidden it

You can only let it run unattended
if the worst case is cheap

Isolated and unambiguous tasks
can run at the same time

Which of these could you safely
stop watching tomorrow?

And here is where I stop
being able to sell you anything

What actually defines each level
is who verifies the work

The instruction that says
“rate this minor” does not hold

We built guardrails so good
they made everything worse

And our “autonomous” mode
is less autonomous than it sounds

Three months, twenty-seven projects,
and it ends with a deletion

We adopted the core.
We didn’t invent it.

So — where are you, and
what’s the one step?