How to Build Agent Memory: Maintenance

In How to Build Agent Memory: Search & Retrieval, we followed an agent as it found a useful memory, checked the evidence, and used it to avoid repeating a debugging investigation.

Now suppose the team changes the code.

The troubleshooting note still exists. Search still finds it. The explanation still sounds plausible. But the condition that made it useful no longer applies.

An agent following that note might waste time investigating a problem the team already fixed. It might even reintroduce a workaround that was deliberately removed. Retrieval can work exactly as designed and still make the next task harder.

This fifth and final post in our series on how to build agent memory covers maintenance: how to keep knowledge useful as the world it describes changes.

A memory can become wrong without anyone touching it

Maintenance gets harder along three dimensions: time, volume, and contradictions.

Time changes the facts. Teams replace dependencies, revise procedures, and make decisions that supersede earlier ones. A memory can be an accurate account of what happened and poor guidance for what to do today.

Volume makes finding the affected knowledge harder. One change might matter to several memories written by different agents for different tasks.

Contradictions appear as those memories overlap. Two notes can describe different versions of the same system without either explaining which version it applies to.

That’s why stale memory can be worse than no memory. Without the note, the agent has to investigate. With an outdated note, it may start from a convincing explanation that sends it in the wrong direction.

The maintenance problem begins with identifying what changed and which memories depend on it.

Keep the evidence behind the summary

One tempting approach is to give an LLM the current summary and the latest event, then ask it to produce an updated summary.

Repeated over time, that can lose details you later need. An exception omitted from the first summary is unavailable to the next update. A pattern that becomes clear across several investigations may be invisible if each investigation has already been compressed.

At Dosu, we retain the original inputs that contributed to a memory, so that an update can reconsider that history.

For our troubleshooting example, the original investigation might contain the dependency version, the failing test, and the conditions that reproduced the bug. When the implementation changes, those details help explain which parts of the note still apply.

Keeping the inputs doesn’t guarantee a correct interpretation. It gives the system evidence it can revisit when its interpretation needs to change.

Give maintenance a manageable scope

Before deciding when to update memory, consider how you will find the memories affected by a change.

This is closely related to retrieval. When new information arrives, you need to locate earlier knowledge about the same subject. A structure that makes relationships explicit can help narrow that search. A more opinionated schema can also make updates cheaper, though it creates constraints on how new knowledge is represented.

Scoping helps too. A change to one team’s workflow should usually begin with that team’s knowledge. A user preference belongs in a different scope from a rule shared across the organization.

Some changes will still cross those boundaries, but scoping lets an update start with the relevant knowledge instead of reconsidering everything the organization has ever learned.

Choose what triggers an update

There are several useful maintenance mechanisms, with different costs and failure modes.

Time-based decay reduces the weight of older memories during retrieval. It’s inexpensive and doesn’t require an LLM, but age is only a rough signal. An old architectural decision may still explain something essential.

Time-to-live (TTL) or least recently used (LRU) eviction keeps the store smaller. TTL expires memories after a set duration, while LRU evicts the least recently used memories when space is needed. Neither age nor infrequent use makes knowledge disposable. A recovery procedure may go unused for a year and still matter when something fails.

Scheduled consolidation, sometimes called “dreaming,” reviews memories for duplicates, conflicts, and opportunities to combine them. The cost can grow quickly if every run examines the entire store. Limiting a run to a relevant subset makes that work more manageable.

Updating on insert uses a new memory as a reason to revisit related ones. If an agent learns a new procedure, the system can check whether the previous procedure needs to be revised or retired.

External events let maintenance respond to work happening outside the agent. A merged PR, resolved support conversation, or closed ticket can change what the organization knows. At Dosu, merged PRs are one of the events we use to update knowledge.

Self-correction lets the system respond when a memory turns out to be wrong during use. The agent may discover that a saved command fails or that the code contradicts an explanation. That discovery needs a way back into maintained knowledge, or the next agent can encounter the same mistake.

These mechanisms can work together. Decay can lower the prominence of older notes while a concrete change triggers a more targeted review. A contradiction discovered during a task can reveal an update that both mechanisms missed.

Decide what is worth maintaining

Maintaining memory resembles maintaining a database index. You pay for work on the write side because you expect it to make future reads more useful.

Some knowledge is expensive to recover. A human explanation of why a decision was made may not be reconstructable from the code at all.

Other knowledge can be recomputed. An agent can rediscover how to start a development environment, although doing so costs time and tokens.

For knowledge that can be recomputed, maintenance has an economic question: how much does it cost to keep this memory current, how often is it used, and how much work does it save?

A frequently used procedure may justify substantial maintenance. An easily reproduced observation that nobody uses may not.

That calculation also needs to account for the cost of getting it wrong. A cheap memory that repeatedly sends agents into unnecessary investigations isn’t saving much.

Measure what happens after the memory is retrieved

A system can store memories, update them, and retrieve them successfully without improving the work an agent does.

Compression alone doesn’t establish token savings in a real task. A short, stale summary may cost more downstream if the agent has to disprove it and start again.

The useful evidence is in the session. Did the agent use the memory? Was it applicable? Did it help the agent reach a correct result, or create a detour? How much work went into checking and correcting it?

Maintenance also needs to be evaluated across changes. A procedure that helps today may fail after the next migration. Looking only at the first successful use misses the part maintenance is supposed to solve.

Forking an agent session into two branches, one with the memory and one without it, could help compare results on the same task. Even without that comparison, reviewing agent sessions can reveal where a memory helped, misled the agent, or went unused.

Make correction part of the system

Memory and knowledge are following the same trajectory as software engineering. Agents can produce code and text faster than people can review it. As that volume grows, we need to shift our focus from inspecting every output to designing and evaluating the automation systems and processes around it.

For memory, those systems need to preserve evidence, check whether knowledge still applies, and detect when an update or correction is needed.

That makes correction an important product capability. People need a practical way to fix what’s wrong, and those corrections need to affect what future agents receive. At Dosu, we’re approaching this by making mistakes easier to correct rather than placing the entire review burden on people upfront.

Return to the troubleshooting note. Once the underlying bug is fixed, the original investigation can remain valuable history. But the guidance the next agent receives should reflect the fix.

That is what maintenance contributes to the system we’ve described throughout this series. Capture preserves a discovery. Storage keeps it accessible. Retrieval brings it into the next task. Maintenance keeps that knowledge aligned with the system it describes.

The value of memory depends on whether it continues to help agents do useful work after things change.