Runbooks for AI Coding Agents: What They Are and How to Write Them

When your team keeps talking a coding agent through the same multi-step procedure, the thing to write is not a longer instruction file. It is one committed file covering that procedure: what starts it, what has to be true before it runs, which commands to run, how to tell they worked, and where the agent should stop and ask a person. That file is a runbook. The writing is the easy half. The work is choosing which procedures earn one and keeping the file true after the code moves.

A runbook holds one procedure, not your standing instructions

A runbook and an agent instruction file do different jobs. An instruction file such as AGENTS.md or CLAUDE.md is standing context: the agent reads it at the start of every session, whatever the task. A runbook is a committed, reviewable file that states one procedure in enough detail that an agent can execute it without a conversation, and the agent loads it only when that procedure comes up. It lives in the repository, next to the code it describes, so a change to the procedure arrives through review like any other change and git log records when the steps last moved.

The distinction matters because standing instructions do less than most teams assume. In Evaluating AGENTS.md, Gloaguen et al. ran four agent-and-model pairings across Claude Code, Codex, and Qwen Code on SWE-bench tasks and on 138 issues from repositories whose developers had committed their own context files. Adding a context file did not generally improve task success rates, and it raised inference cost by over 20% on average. LLM-generated files had a marginal negative effect and developer-written files a marginal positive one, and neither difference was statistically significant. The agents did follow the instructions they were given. What failed to help was the repository overview that makes up much of a typical file. The authors conclude that a context file should hold only specific instructions the codebase does not already supply, such as non-standard conventions.

That result is the case for a runbook. Agents follow precise instructions, and spending every session's context on general orientation does not pay for itself. A runbook keeps the part that works, an exact procedure, and loads it only for the task that needs it.

Where it goes depends on the tool your team runs. In Claude Code, a project skill is a directory holding a SKILL.md, at .claude/skills/<skill-name>/SKILL.md, committed so everyone working in that repository gets it. Claude Code keeps every skill's description in context so the agent knows what is available, and loads the full body only when the skill is invoked. Other agents read other files, so check your tool's current documentation for where it looks and when it loads what it finds.

Anthropic's own guidance draws the same line. Claude Code's memory documentation says a CLAUDE.md entry should be something that holds in every session, such as build commands, conventions, and project layout, and that an entry which is a multi-step procedure, or which only matters for one part of the codebase, belongs in a skill or a path-scoped rule instead. Instruction files are read at the start of every session, so a forty-step deploy sequence pasted into one spends context on every task that will never deploy anything.

Keeping the procedure separate also makes it reviewable on its own terms. A reviewer can check six ordered steps against the code they describe without reading past the project's standing conventions to find them. For the longer comparison of which instruction file each tool reads, see our guide to AGENTS.md, CLAUDE.md, and .cursorrules.

Let session evidence pick the procedure

Write a runbook when your own sessions show the same procedure being rediscovered, not when a procedure feels important. The evidence is in the transcripts: a correction an engineer has given more than once, an environment failure that recurs across unrelated tickets, a command sequence agents keep assembling in the wrong order.

Dosu's review of 107 coding sessions on Vercel's AI SDK found 68 facts that recurred across tickets, and the clearest one was a build requirement that surfaced in 59 sessions across 40 tickets. In one inspected replay, the test run could not import @ai-sdk/provider-utils because the workspace package was present but its compiled output was missing. The agent built the workspace dependencies and the tests ran.

Note what that repository had written down before any of those sessions ran. Its own AGENTS.md listed pnpm install followed by pnpm build, so the ordering was documented. What no file supplied was the connection between a failed import and a missing dist/ directory, which is the thing each agent had to work out at the moment it was stuck. A runbook earns its place when it closes that kind of gap, by naming the symptom alongside the recovery, rather than restating a step a contributor guide already lists. Those counts come from model-extracted findings over fresh checkouts on selected tickets, and the study did not verify a failure-and-fix sequence in every counted session.

A second kind of candidate never shows up as a repeated correction. Some procedures carry an ordering, an approval, or a notification step that no amount of reading the code would reveal, such as rotating a credential or running a destructive migration against a shared environment. Those are worth writing before the first incident, with the steps the agent may take stated separately from the decisions a person has to make.

If you do not have a way to read your team's transcripts yet, our guide to analyzing Claude Code and Codex session logs covers locating and redacting them first.

A stale runbook keeps being followed

An instruction file that has drifted gets ignored or argued with. A runbook that has drifted gets executed, because a committed file carrying exact commands reads as more authoritative than anything the agent infers from the code. Reviewing a runbook and retiring it are part of owning one.

Review a runbook when a pull request changes the commands, services, configuration, or expected output it describes. Check it against the current code rather than against its last edit date, and look for four specific kinds of drift: a flag or configuration value that makes a command wrong, a command that still succeeds but no longer produces the result the runbook claims, a changed order, and a copyable example that no longer matches the repository. Where it is safe, run the steps in the environment the runbook names and compare what happens against what it says should happen. A passing example does not verify the failure branches.

Retire a runbook deliberately. Delete the file from the location the agent reads, update anything that pointed at it, and send the replacement through review. If the old procedure still applies somewhere, narrow its trigger to name that environment rather than leaving both versions discoverable. Version history keeps the record for anyone who needs to see what the procedure used to be. Our guide to keeping agent instruction files updated covers detecting this drift across a repository rather than one file at a time.

What a runbook will not do for you

A runbook supplies steps for a procedure someone has worked out. It will not help the agent decide whether this is the right procedure for the task in front of it, and it does nothing for the larger cost of working out what the task needs. In Dosu's replay of 112 coding-agent sessions against the cal.diy repository, 46% of modeled dollars were spent before the first line of code changed, with Claude Code at 43% and Codex at 52%. Those figures measure orientation in those sessions at standard API rates, and they do not establish how much of that work any runbook could remove.

Nor is a runbook a control. Claude Code's documentation is explicit that instruction files and retained notes are context rather than enforced configuration, and points at a PreToolUse hook when an action has to be blocked regardless of what the agent decides. If a step must never run in production, the hook is the mechanism and the runbook is the explanation.

The last limit is scope. A procedure whose correct response varies with the service, the logs, or the state of the incident resists a fixed sequence, and compressing it into one anyway produces a file that sends the agent confidently down the wrong path. Write that one as a trigger, a list of what to gather, and an instruction to bring it to a person.

FAQ

How is a runbook different from a CLAUDE.md or AGENTS.md entry?

Scope and loading. A CLAUDE.md or AGENTS.md entry holds standing context the agent should carry into every task in that repository, and it is read at the start of the session. A runbook holds the ordered steps for one task, including its trigger and its success check, and a tool like Claude Code loads the body when the procedure is invoked. Anthropic's guidance is to move a multi-step procedure out of the always-loaded file for that reason.

How many times does something need to repeat before it earns a runbook?

There is no threshold worth defending. Look for the same corrective turn or the same environment failure across separate tasks and separate engineers, which is a stronger signal than a raw count from one person's week. A procedure can also earn a runbook with no repetition at all when its steps carry an order or an approval the agent could not infer from the code.

Who should review a runbook before it is committed?

Two people, for two different things. Whoever owns the procedure checks the commands, the access they need, and the expected output against the current repository. Whoever runs the agent checks that the trigger and the stop conditions are unambiguous. Run the safe steps during review rather than approving the wording.

What should happen when an agent hits an out-of-scope boundary mid-task?

The agent should stop before taking the action, report what it completed, what condition it hit, and what remains, then ask for direction. Write that handoff into the runbook explicitly. A procedure that is silent about its edges invites the agent to treat an unlisted case as permission to proceed.

Start from a correction you have given twice

Pick one correction you have given an agent more than once this month and check whether it recurs for anyone else. You can read your own transcripts directly, or run Decant, Dosu's open source session analytics tool, with npx @dosu/decant@latest, which indexes the agent session logs on your machine, Claude Code and Codex among them. Decant makes no outbound network calls at runtime, apart from an update check that stays off until you turn it on, but it does write prompts, tool inputs, and tool output into an unencrypted SQLite archive at ~/.decant/decant.db, which will include source code and anything pasted into a session, so treat that file as subject to whatever retention policy covers your transcripts. When the same procedure shows up twice, write its trigger and its steps, add a success check and a stopping condition, and send it through review.

Committing the file solves review and makes the procedure available to everyone working in that repository. What it does not solve is the knowledge that never becomes a procedure: the decision and its rationale, the constraint that explains why the steps are in that order. Dosu is knowledge infrastructure for engineering teams and the coding agents they work with. Through the Dosu MCP server, an agent retrieves that context with read_knowledge and records it with write_knowledge, and a note anchored to a repository and branch is visible to other Library members reading the same repository and branch, whichever MCP-compatible client their agent runs in. Create a Library and connect the repository. Creating a Library requires an organization Owner or Admin role. Then point your agent at it with npx @dosu/cli setup.

Sources

Claude Code behavior above is from Anthropic's Claude Code skills and memory documentation, checked November 5, 2026. The AGENTS.md format is described by its published specification. The context-file results are from Gloaguen et al., "Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?" (arXiv 2602.11988, v3). Session measurements are from Dosu's own published analyses, with their stated conditions. Dosu MCP behavior is from the Dosu product documentation.