Coding Agent Metrics: What to Track Beyond Lines of Code

A weekly agent report is easy to assemble from the counters every tool exports: pull requests opened, lines written, suggestions accepted. What none of them record is where each session spent its budget before that code existed, and how many times an engineer stepped in to get it there. When Dosu measured its own agent sessions, writing code was a minority of both the bill and the clock. The metrics below describe the majority.

Output counters leave session costs unexplained

Claude Code's OpenTelemetry export emits claude_code.lines_of_code.count with a type of added or removed, claude_code.commit.count, claude_code.pull_request.count, and claude_code.code_edit_tool.decision, which records accept or reject for Edit, Write, and NotebookEdit. Those four are the simplest numbers in the stream to put on a slide and the weakest evidence that a session went well. An accepted edit can be rewritten twenty minutes later. A rejected one can be the moment someone steered the agent back onto a working path.

Dosu's own measurement shows how little of the work those counters cover. Replaying 60 pull requests from the public cal.diy repository as 112 agent sessions, with Claude Code on Claude Opus 4.8 and Codex on GPT-5.5, writing code accounted for 26% of modeled dollars and 19% of active time, while planning and gathering context took 67% of the dollars and 65% of the time. That code bucket is wider than the diff, since it covers the build, test, and git commands around the edits as well as the edits themselves. Claude Code moved about 198 tokens through its context window for every token it generated, and Codex 134. Those costs are modeled at standard API rates, eight runs where an agent found and copied the original pull request were excluded, and no session was scored for correctness. They measure spend, not which agent writes better code.

A session leaves three records, and they answer different questions

Every metric below comes from one of three places. The agent's own telemetry gives you counters and structured events across a fleet. The session transcript on disk gives you the prompts, tool arguments, and per-call token counts that the counters leave out. Your CI and review history gives you what happened to the change afterward.

Claude Code redacts content by default, but the controls interact. With prompt logging enabled, set OTEL_LOG_ASSISTANT_RESPONSES=0 to keep response events redacted. Detailed beta traces and raw API-body logging need a separate content review. Redacted telemetry supports some metrics below, while transcript analysis reads sensitive content locally. For a comparison of the three collection paths and what each one costs you, see Best Tools for Coding Agent Observability.

Where the session budget went

Context and planning share of spend

The share of a session's tokens, modeled cost, or active time spent reading and planning rather than editing. Decant, Dosu's Apache 2.0 reader for local session logs, sorts each session's generation, tool calls, modeled cost, and active time into context, planning, code, and communicating, and reports the split with decant economics. Its classification is deliberately conservative: a compound shell command counts as context only when every part of it is read-only, and an unrecognized or runtime-assembled command counts as code. The bucket costs are proportional allocations that reconcile to the session total rather than provider charges.

A single number means nothing without a baseline for comparable work. In the cal.diy study, Claude Code's planning and context share was 71% of spend on self-contained tasks and 81% on cross-cutting ones, so the share rises with how much of the codebase the task requires the agent to understand. Read the contents of the bucket before reacting to its size. Searching a package for the first time is the job. Searching the same package on every ticket is a finding.

Orienting share, measured against the first edit

Orienting is everything before the first file edit, and it is the fastest-burning part of a session because it is mostly low-latency reads. Decant marks the boundary at the first structured edit, including apply_patch inside a Codex exec program, plus narrow shell patterns such as git apply and sed -i. A session with no edits counts entirely as orientation, and the classifier prefers to miss a weak signal over marking a false boundary. The economics command does not print this split on its own.

Measured across those 112 sessions, 46% of modeled dollars went to orienting, at 43% for Claude Code and 52% for Codex, and the average session ran 4.2 minutes with 1.3 of them spent before any edit. A session that drifts well past its tier's baseline is worth opening. Read what the agent searched for against the ticket and the entry point it was given, rather than assuming which of the two was at fault.

Peak context-window occupancy

Decant computes window occupancy per model call as input plus cache-read plus cache-creation tokens, and takes the peak as the largest observed call. Where the window size is inferred from the model rather than recorded by the agent, Decant marks it inferred and grows it a tier when a call exceeds it, so a peak never reads above 100%. High peak occupancy is a reason to inspect the session's recorded compaction boundaries and what happened afterward. Check whether required context was lost or work repeated before treating compaction as a problem. What a coding agent loses at compaction covers that case.

Facts the next session has to rediscover

The expensive repetition happens between sessions, not inside one. Across 107 Claude Code and Codex sessions on vercel/ai, 28% of the extracted facts appeared on a different task, and one build requirement showed up in 59 sessions across 40 tickets. Keep that across-task figure separate from the study's broader recurrence bar, which counts a fact seen in at least two distinct sessions and so includes two agents attempting the same ticket. The analysis deliberately separates repetition within one task from knowledge that reappeared on different tasks. All of it is model-extracted, and each fact counts once. To look for the same thing yourself, use decant files --op read and full-text search over tool arguments to find the candidate sessions, then read those sessions and decide whether they arrived at the same fact. Repeated paths locate the sessions worth opening rather than establishing a repeated discovery, and without --op the command counts edits and writes alongside reads. A fact that three sessions each paid to derive belongs somewhere the fourth one can read.

What the handoff cost a person

Corrective turns per task

Count messages that redirect an agent before its output is usable, excluding the opening request and routine approvals. A merged pull request hides this number completely, which is the reason to collect it. Read each correction alongside the original request and repository guidance to distinguish missing information from information the agent overlooked. Compare similar tasks and check the resulting patches before interpreting a change in the count.

Share of tasks finished without a corrective turn

The inverse view, reported separately for small fixes, scoped features, and cross-cutting changes. One team-wide rate moves when the task mix moves, which makes it unreadable. Segmenting by tier makes a drop mean something.

Failed tool-call rate

Errored tool calls over all tool calls, grouped by tool. The claude_code.tool_result event carries tool_name, success, duration_ms, and tool_result_size_bytes with no content attached, so a rate per tool survives a fully redacted telemetry configuration. Grouping Bash failures by the command that failed is a further step, and the command text reaches you only through OTEL_LOG_TOOL_DETAILS or a local transcript. From transcripts, Decant computes a success_rate from recorded tool results, excluding Codex calls whose results the agent never recorded.

Group the failures by command before acting, which needs the collection step above. Malformed arguments are the agent's to fix. For a failing build or test command, run it on the base commit and on the agent's checkout with the same setup before deciding the repository was already broken, since the agent's own changes can be what breaks it.

Reading a bad number: the ticket, the repository, or the agent

Every signal above has at least two explanations, and comprehension scope is what lets you compare like with like while you work out which one applies. A second Dosu study makes the case for stratifying by scope before reading any cost number: across 49 vercel/ai pull requests attempted by both agents, Codex's estimated total was 0.62 times Claude Code's, but the ratio was 1.28 on low-scope tickets and 0.45 on high-scope ones. The pooled ratio reads as a Codex saving, while on the 29 low- and mid-scope tickets the two agents came out level or Codex cost slightly more. These are costs of attempts at pinned prices, with no scoring for correctness.

So set baselines per tier, then work through the cheap explanations first. Check AGENTS.md or CLAUDE.md for paths that moved. Run the build and test commands yourself. Check whether the ticket named the affected package. Change the agent's instructions last, once the repository and the request have both been ruled out.

The four to instrument first

You do not need a dashboard to start. Four measures cover most of what the counters miss.

• Context and planning share of spend. Run decant economics for the split across your indexed sessions. It takes no arguments and reports over the whole archive, so open individual sessions with decant ls and decant show when you need to read one tier. The command needs no exporter, no collector, and no change to how anyone works.
• Orienting share. Mark the first edit in each session and sum the cost before it. Treat the number as a budget per tier rather than per task.
• Corrective turns per task. Count them from the transcript. Corrective turns are the one metric here that a fully automated pipeline will not hand you, and reading the corrections themselves is what makes the count interpretable.
• Failed tool-call rate. Group by command, which needs tool-detail logging or the transcript. Compare a failing build or test command across the base and changed commits before deciding whose problem it is.

FAQ

Is a high context share always bad?

No. A cross-cutting change requires more reading than a one-file fix, and in the cal.diy study the share rose from 71% to 81% of Claude Code's spend with scope. What makes it a finding is repetition: the same files and the same searches across tasks that are supposed to be independent.

Can I collect these without logging prompts?

Partly. Claude Code's counters and its tool_result events carry no prompt or tool content unless you set the content flags, so a failed-tool-call rate per tool, session duration, and the token split by type (input, output, cacheRead, cacheCreation) are available from a redacted stream. That type split is not the context and planning split: sorting spend into activities, and grouping failures by the command that failed, both need the tool text, through OTEL_LOG_TOOL_DETAILS or a local transcript. The identity attributes are a separate question, since every event carries account identifiers and a user.email whenever Claude Code has an email available from the sign-in. Reading transcripts instead keeps content and identity on the developer's own machine, which is its own tradeoff to weigh.

What if our agent does not export any of this?

Fall back to the transcript. Decant parses Claude Code, Codex, Gemini CLI, and OpenCode logs from their default directories. Cursor is not in that set today, so a Cursor rollout needs its own verified collection path. Record what you cannot collect as unavailable rather than as zero.

Read your own archive before you change anything

Run decant economics for the context, planning, code, and communicating split, then decant files --op read for the paths your sessions read most. Both report over every session the archive holds, with no date, repository, or task filter, so read them as an archive-wide baseline and use decant ls and decant show to open the sessions behind a number that looks wrong.

Then look at what those paths have in common. Connect that repository to Dosu, then set up the Dosu MCP server with npx @dosu/cli setup to configure your agent's connection. Have agents save durable findings with write_knowledge and retrieve them with read_knowledge before they start searching. Compare matched completed tasks before and after, counting the capture and retrieval calls too, to see whether total cost and correctness changed.