The Four Costs of Coding Agent Memory and How to Measure Them

Adding memory to a coding agent introduces four costs: time to retrieve information, tokens to use it, work to save it, and human effort to keep it accurate. Retrieval speed is only one of them, so a latency test alone will not tell you what a memory layer costs.

• Retrieval time is how long the agent waits to fetch saved information.
• Token cost is the extra model usage from including that information in requests.
• Saving cost is the processing needed to capture and store what a session learned.
• Maintenance effort is the time people spend reviewing, correcting, and removing saved information.

We know of no published measurement that puts all four in the same units, so treat any ordering, including ours, as a hypothesis to test on your own sessions. Start by measuring what your agents spend on a task today. With that baseline you can tell whether memory saves more time and money than it costs.

For a sense of that baseline, we replayed 60 pull requests from a public repository as 112 Claude Code and Codex sessions and measured them with Decant. Planning and context gathering accounted for 67% of estimated dollars and 65% of active time, averaged across both agents. Writing code, including edits and the surrounding build, test, and git commands, accounted for 26% of dollars. The study did not measure memory retrieval, so these results describe session costs, not the overhead or savings a memory service would add.

Retrieval latency depends on when memory loads

A retrieval call delays the agent only when the agent needs the result before it can continue. An on-demand lookup over MCP usually costs a model call to choose the tool, the fetch itself, and another model call that can use what came back. Time that whole round trip against the model calls in the same turn. A database benchmark of the store leaves out both model calls.

Check whether memory loads at session start or on demand. Claude Code loads CLAUDE.md and auto memory at the start of every conversation. A subdirectory's CLAUDE.md loads only when Claude reads, writes, or edits a file there, and path-scoped rules load when an operation matches their paths. Eagerly loaded memory is on the critical path of every turn from the start of the session. Scoped memory loads at the turn that touches it, and from then on it is part of the conversation the agent re-sends.

Memory in context is billed on every request that carries it

Memory text travels with the request. An always-loaded instruction file is charged as input tokens on each request that carries it, for the length of the session. On-demand retrieval delays that cost until the content is fetched. Once retrieved, the content can remain in conversation history and incur input or cache-read charges on later requests until the agent removes or compacts it. The tool definition may also occupy context before the first fetch. Claude Code defers MCP tool definitions by default on supported models and loads only tool names and server instructions at session start, so there the full definition enters the conversation when the agent first looks the tool up. Count both the calls the agent makes and the tool definitions it carries on turns when it makes none.

Prompt caching lowers that charge without removing it. Anthropic's caching documentation prices a 5-minute cache write at 1.25 times base input tokens and a cache read at 0.1 times base input, with lower read multipliers on several current models. Reuse requires a prefix that matches exactly, clears the model's minimum cacheable length, and is still in the cache, which defaults to a 5-minute lifetime refreshed each time it is used.

Stable, eagerly loaded context near the front of the prompt can be cheap across a session. Memory rewritten between requests costs more, and how much more depends on where your cache breakpoint sits. Put the volatile part after a stable breakpoint and the prefix still earns a cache read while the changed remainder is billed as ordinary input. Anthropic's documentation lists the reverse as a common mistake. Put the breakpoint on a block that changes every request, such as one holding a timestamp, and in the documentation's words, "You pay for a fresh cache write on every request and never get a read."

As a worked example at those default rates, take a 2,000-token instruction file carried on 40 requests inside the cache lifetime. Uncached, the file bills as 80,000 base input tokens. Behind a stable breakpoint, one write and 39 reads bill as 10,300. With the breakpoint on content that changes every request, all 40 requests pay the write rate, which comes to 100,000.

Keep session-start instructions short and stable, and confirm the result in your cache-read and cache-write counts, because the prefix that carries a short file has to match exactly and clear a model-specific minimum length before it earns a cache read. Claude Code's documentation says to target under 200 lines per CLAUDE.md, because longer files consume more context and reduce adherence. Imports do not help, since imported files load at launch too. For auto memory, Claude Code loads only the first 200 lines or 25KB of MEMORY.md at the start of a conversation and leaves anything past that threshold out.

Move rarely needed facts behind on-demand access. A path-scoped rule in Claude Code loads when the agent touches a matching file, so an instruction for one service stays out of context until the agent works there, and out of the sessions that never do. Retrieval spends tool-call tokens on the turn that needs the fact, and the fetched text then stays in the history the agent re-sends on later requests. What you save is the turns before the fetch.

Where capture runs decides who waits

Writing memory is a separate cost from reading it. The agent decides what is worth saving and writes it. When the new note contradicts an older one, the agent also reads the old entry and reconciles the two. Running that work in the background does not make it free.

Capture inside an active session adds model calls and file operations before the agent returns to the developer's task. Capture after the session ends keeps those operations off the developer's path and opens a window where the next session starts without the new information. Move capture off the developer's turn where your workflow allows, and accept that updates must finish before the next session can rely on them.

Before you commit to a memory layer, find out when it captures, whether a developer waits for the write, and how it resolves conflicting notes.

Wrong memory costs rework that a latency test never sees

An instruction that points the agent at a retired database adapter sends it to the wrong file, and you pay for the edit, the review that catches it, and the rework. Faster retrieval cannot fix a fact that is no longer true, so count rework in the same budget as latency.

STALE, a benchmark published in May 2026, tests whether agents notice that their memories are no longer valid. The benchmark checks whether an agent detects a changed fact, rejects a question that presumes the old one, and applies the update in later behavior. The best frontier model it evaluated, Gemini-3.1-pro, reached 55.2% overall accuracy across those tests. The researchers describe a gap between retrieving newer evidence and acting on it.

Every conflict in STALE is implicit, meaning nothing later in the conversation says the old memory is outdated. A Type I conflict implies a new value for the same attribute, such as a repository moving to a new runtime version. A Type II conflict changes a related attribute, and the change cascades to invalidate the memory. Database access moving behind a service, for example, retires a note about editing queries in the caller. Nearly every system the paper tested scored lower on Type II than on Type I.

Someone still has to review what memory saves

When a service gets renamed, someone has to notice that the saved note still names the old one. A policy keeps that work bounded:

• Filter routine session detail before it reaches shared memory.
• When a new fact conflicts with a stored one, decide which wins and mark the loser so the agent cannot retrieve both as current.
• Give fast-moving facts a review trigger, such as a change to the file they describe.
• Delete records you are obliged to remove, such as secrets or personal data, and prune what has stopped being useful.
• Name an owner and a recurring slot for review. The owner reviews flagged conflicts, prunes obsolete guidance, and corrects memory after the code changes.

How to prune stale agent memory covers firing a retirement off the change that made a fact false. Tools can move this work around without removing it, so put the review hours in your cost estimate.

Where each kind of memory tool puts the cost

Always-loaded instruction files fit rules the agent needs in most sessions. They load at session start by design, you maintain them yourself, and review is part of the price. To decide which facts belong in one, sort them by who must approve them, whether they must outlive one laptop, and how often they change.

On-demand retrieval fits facts the agent needs less often. Mem0, Zep, and Cognee are built for this kind of retrieval and are worth evaluating if you are assembling a memory integration. Each one exposes memory through an SDK or API that you integrate. Check its current documentation for how it writes, retrieves, and expires facts, because those choices determine your upkeep, and expect to decide what gets captured and who fixes a bad record unless the documentation shows otherwise. If you are choosing an MCP memory server, compare servers by where they keep the store and who can reach it.

A hosted memory API runs the store for you and leaves the cost structure in place. The retrieval call, the injected context, and the checking of stored facts all remain. An API can sit behind an MCP tool, so evaluate how the agent reaches memory separately from who operates it.

Dosu is in the last row because the upkeep is the part it takes on. Dosu generates knowledge from the Sources your team connects, such as GitHub, plus Slack and Notion on the Pro and Enterprise plans. Coding agents reach that knowledge on demand through the Dosu MCP Server, so it is not loaded into every session.

Dosu checks for stale knowledge outside the coding session. With a Monitor enabled on a Source, Dosu checks non-draft pull requests in the paths it monitors against published Documents and drafts a targeted edit for anything that looks stale. Knowledge Reviews tie review to pull requests, so the load tracks code changes. A proposed edit waits for a person to Accept, Edit, or Decline, unless Auto-Accept Review is on for that Library, in which case updates nobody actioned publish when the PR merges. Notes that agents write mid-session through the MCP Server save directly, without waiting in Review. The upkeep does not disappear. Dosu moves the check onto the pull request that caused the drift.

FAQ

Does adding memory slow down an AI agent?

An on-demand lookup adds latency on the turn where the agent makes it, and none on the turns where it does not. The retrieved text then sits in conversation history and is re-sent on later requests until a compaction clears it, so on-demand loading moves when the token cost starts and does not end it at the fetch. An eagerly loaded memory file adds no retrieval call at all and carries its contents from the first request onward. The two load patterns cost you in different places, so measure the one you are buying.

Does prompt caching eliminate the token cost of memory?

Caching reduces it without removing it. Anthropic prices cache reads at 0.1 times base input tokens by default, with lower rates on several current models, and prices cache writes above base, so a cached prefix is cheaper than an uncached one and is not free. A memory file that changes before the cache breakpoint can prevent the hit entirely, so check your cache-read and cache-write counts to see whether the file is cached. In Claude Code, editing your project-root or user CLAUDE.md mid-session leaves the cache intact, and the edit takes effect at the next /clear, /compact, or restart.

How is wrong memory different from stale memory?

A wrong memory was inaccurate when it was captured, such as a note naming the wrong test command. A stale memory was true and has been overtaken, such as the command your repository used before a migration. You fix the first by correcting it and the second by detecting the change that invalidated it, which is why a store needs both review and triggers.

Does eviction fix a growing memory store?

Eviction caps size, and it cannot tell you which surviving facts still match the codebase. Deleting unrelated old notes does nothing about a service that got renamed last month. Control what enters the store, and invalidate facts when the thing they describe changes.

Measure your baseline before you buy a memory layer

A vendor benchmark cannot establish overhead for your repository, your tasks, and your agent settings, so start with your own sessions. Run Decant over your Claude Code or Codex transcripts with npx @dosu/decant@latest. Decant reads the logs locally and reports token spend, estimated cost, context usage, and which files and tools the agent touched. Record those numbers and how much of the session runs before the first file edit. Decant makes no outbound network calls at runtime, though it does keep an unencrypted SQLite archive at ~/.decant/decant.db, so check that against your rules for storing code before you import.

Without a baseline you cannot attribute a later change to memory. Once you have one, enable memory and run comparable tasks in the same repository, with the same agent and model. Compare the two sets of sessions on:

• input and output tokens per session, including cached input tokens where the agent reports them
• time to first useful edit, since an early change the agent later throws away did not shorten the work
• how often the agent re-asks for a repository fact that memory should have supplied
• whether what the agent retrieved was right

Decant covers the token numbers and estimates active time before the first detected file edit. A person still has to judge which edits were useful and which questions repeated, so keep those notes beside the session data.

Then look for the tokens your agents spend re-deriving context your team has found before, because that context is worth storing once. Connect a repository to Dosu and turn on a Monitor, so proposed updates to that knowledge arrive on the pull requests that change the code. Existing Dosu users can run the same comparison against their current Library.