How to Build Agent Memory: Knowledge Capture

This post is based on our Agent Memory: What Information Is Worth Remembering? webinar. Watch the recording or read the full transcript there.

Your agent is debugging something that should work. You think it should work. The model agrees. Forty minutes, several dead ends, and a few human corrections later, you finally understand why it doesn't.

The problem is fixed. But when another agent encounters the same issue tomorrow, what keeps it from repeating that investigation?

In the first post in this series, we broke agent memory into four components: representation and storage, extraction, retrieval, and maintenance. In the second, we explored where knowledge lives and distinguished historical artifacts from what an agent should currently believe.

This post is about the write path: how useful discoveries become knowledge that future agents can inherit.

Capture what was hard to learn

Troubleshooting is one of the clearest examples of knowledge worth capturing. The final answer might be small, but getting there required experiments, rejected explanations, and information the model didn't have.

Reverse engineering is similar. An agent probes an undocumented system, tests assumptions, and gradually figures out how it behaves. The discoveries along the way can be expensive to reproduce.

Human input can be even harder to recover. If someone explains why a seemingly unnecessary behavior must stay, that reasoning may never appear in the code. Without a record, the next agent may confidently remove it.

These are strong candidates for memory because they help future agents avoid repeating difficult work. A fact sitting in an obvious configuration file is a weaker candidate. Reading the file may be cheaper than creating and maintaining another representation of it.

As we discussed in the previous post, knowledge behaves like a cache. But a write costs more than storage. Extracting facts, creating embeddings, building relationships, and keeping the result current all take work. Adding more memories can also introduce noise into retrieval.

The calculation is whether expected future use justifies those costs. Something expensive to discover may be worth saving after one investigation. Something cheap but needed constantly may also be worth saving.

Agent session data gives us a useful starting point: how many tool calls did the investigation require? How much time did it take? Where did a human have to intervene? Those measurements help estimate rediscovery cost, though they don't tell us whether the result will matter again.

The thing to capture is the discovery that could change future work. “We debugged the integration” doesn't help much. The unexpected behavior, the conditions that trigger it, and the evidence behind the fix do.

Five questions before writing knowledge

At Dosu, we're still iterating on the rubric we use to decide what deserves to become knowledge. These are the five questions we keep coming back to.

1. Would this change a future action?

Would knowing this prevent a mistake, eliminate a dead end, or change how an agent approaches a task?

In the debugging example, the valuable learning might be that a normally reasonable assumption fails under a particular condition. Capturing that exception gives the next agent a better starting point.

2. What makes us believe it's true?

Information has a source, and that source matters. A passing test, a direct observation, a human explanation, and an agent's working theory provide different kinds of support.

Even someone who built the system can misremember how it works. An agent can also reach a plausible conclusion before the investigation is finished.

Capture the supporting evidence with the claim. Link to the relevant session, code, discussion, or experiment so a future agent can check the reasoning and recover details that the summary leaves out. Session traces are especially useful here: they can preserve what the human wanted, which approaches failed, and how the conclusion emerged.

3. Where is it true?

A memory can be accurate and still lead an agent astray if it applies in the wrong context.

I ran into this while working on a side project with several parallel coding agents. One agent saved memories about changes on its branch. Another picked them up and started reasoning about functionality that didn't exist on its own branch yet.

The problem was scope. Knowledge about work in progress had escaped the context where it was valid.

The same issue appears across organizations. One person's preference isn't automatically team policy. A convention in one repository may not apply to another. At write time, ask whether the claim belongs to a session, branch, repository, team, or the broader organization.

4. What happens if it's wrong?

Some memories have a much larger downside than others.

An incorrect formatting preference may create a small cleanup task. Incorrect guidance about authentication or data deletion can cause substantial damage. High-value knowledge can carry high risk precisely because agents are likely to act on it.

The consequences of error should affect how much evidence you require and how broadly you allow the memory to apply.

5. How hard would it be to discover again?

Return to the economics. Could the next agent read one file, or would it need to repeat a long investigation? Would it need access to a person or system that might no longer be available?

Time, tokens, and tool calls help answer this question. So does noticing which mistakes the agent made along the way. A short fact that prevents a recurring wrong turn can be more valuable than a detailed description of an easily inspected system.

How much structure should a memory have?

Once you've decided something is worth capturing, you still have to decide how to express it.

A fixed schema makes extraction more predictable. If every memory has a category and a defined set of fields, the system has less to figure out when writing and reading it.

But the schema also shapes what the system notices. If you're only extracting preferences, decisions, and entities, an unexpected troubleshooting lesson may not fit anywhere. Information outside your categories can disappear before you know you'll need it.

Flexible notes preserve more nuance, while raw session histories preserve still more context. That flexibility moves work into retrieval: a future agent has to search and interpret the material.

A practical starting point is a concise knowledge layer that points back to the original artifacts. Give each memory enough context to explain the discovery, its scope, and its supporting evidence. Add more structure when you see a recurring need.

Keeping the source matters because every extraction loses detail. You want a future agent to be able to revisit the original evidence when the summary isn't enough.

How knowledge gets captured

There are two broad ways to populate that knowledge layer.

With active capture, an agent explicitly writes a memory during its work. It discovers a gotcha, receives an important correction, or finishes a difficult investigation and saves the learning.

With background capture, a separate process examines session histories, code reviews, or conversations for information that might help future work.

Active capture lets the agent record a discovery while the context is available. Background capture creates an opportunity to find useful information that wasn't saved at the time. A system can use both, but either approach still needs to answer the five questions above.

There are open questions here, including whether knowledge written by one model will be useful to another. The amount of explanation an agent needs, and what it finds difficult to rediscover, can change as models improve.

For now, a useful place to start is the next difficult debugging session. Capture what you learned, where it applies, and why you believe it. Give the next agent enough information to avoid the same dead ends.

That's the write side of what we're building at Dosu. Next in the series, we'll look at the other half: how an agent finds and uses that knowledge when it needs it.