Agent Memory: Where Does Knowledge Live?

Episode 2 of Dosu's agent memory series goes deep on the first of the four modules: what a stored memory actually looks like on disk. Taylor Dolezal and Devin Stein work through the three logical representations the survey paper lays out — token-level sequence, graph and tree topology, and heterogeneous composite — then get into storage tiers, why versioning and branching matter more than the literature suggests, and why the practical advice for most teams is still to put it in the Git repo and see where it breaks. The series works through Are We Ready for an Agent-Native Memory System? (Zhou et al., 2026); episode 1 covered how memory decomposes into four modules.

Transcript

Taylor Dolezal, Head of Open Source, Dosu: All right, well, hello, everybody. Welcome. Let's get started and jump into some more agent memory conversations. Today we're going to be talking about: where does knowledge live? We've got quite a bit to get into here.

My name is Taylor Dolezal. I'm the head of open source here at Dosu, and I'm joined by Devin. I'll throw it to you for your intro — if you haven't met us before, happy to meet you.

Devin Stein, Founder and CEO, Dosu: Hi everyone. Devin Stein, CEO and founder of Dosu. Thanks for being here. Very excited to be talking about where knowledge lives.

Taylor: I'm excited to find out. I feel like I don't always know myself.

If you haven't caught up with us before, we had a previous episode where we jumped into how memory decomposes into four different modules. We'll share out the paper that we're detailing and diving into today — you can check the notes in the video.

But today, with this paper, we're going to go deep on that first module: what does a stored memory look like on disk? So in this session we're covering storage and format. In future sessions we're going to cover retrieval and search — read operations. Then we're going to cover capture and write. After that, update and delete. So you can see the CRUD we're spelling out there, if you're familiar with database operations. Then we're going to get into benchmarks, evals, and measurement in a future episode. And then we're going to end this series with integrations.

Logical representation vs physical storage

Taylor: So to kick things off and set the stage: Devin, what would you say the difference is between logical representation and physical storage, and why does this paper keep those separate?

Devin: Good question. The paper breaks up how to think about memory logically — how is memory structured, what are the primitives you're using to describe a memory, how do you relate memories to each other, and what combinations of those there can be — from, physically, how is that piece of information stored and then later retrieved.

I think it's useful because, just from an abstractions perspective, the storage component can be swappable. So in my opinion the logical representation is probably the most important part, as we figure out what the best systems under the hood are to actually store this information.

Token-level sequence: plain text and AGENTS.md

Taylor: Gotcha. I think that brings us to our first section, really focusing on format. There are three logical representations, and maybe a little bit of a bonus round as well. Those are token-level sequence, graph and tree topology, and heterogeneous composite, which we'll detail. And then the bonus round is the storage side.

So to kick things off with token-level sequence — Devin, could you talk about what that is and help frame it for us?

Devin: Of course. Token-level sequence is a fancy way the paper describes something as simple as possible: plain text, or just pure tokens. So this can be a piece of information that's put into a system prompt. That could be your user memory in Claude, or AGENTS.md, which everyone's familiar with, that gets shoved into the beginning of every session so the agent is aware of it. That's just a token-level sequence format. There isn't a lot of structure to it — it's basically a blob of text.

Another common format that's used is just raw transcript histories. That's also a simple format of token-level sequence memory. So that's searching through your previous conversations, like Claude and Codex do this pretty effectively. There are open source tools like DejaVu that help you do this on your coding agent sessions as well.

And then the other slightly more abstract but still token-level sequence example is facts, where you're doing some processing of information from transcripts, trying to extract key memories and facts from those conversations. I think ChatGPT does this pretty well, where it'll remember specific details about you — or at least try to. We can maybe talk about experiences with memory in those products.

But token-level sequencing is really just about flat structure, pure text representation. And importantly, as a contrast to what we'll get into: there are no relationships between them. It's sort of a bucket of text.

Taylor: Gotcha. So this is mostly unstructured text, like you said. It might be something like markdown — or do you know if there are any other forms on that front, where something might be token-level sequence but not as readable as markdown?

Devin: Yeah, so the paper also buckets actual token- or embedding-based memory in here — so parametric memory, where you can think of it as: you have your input tokens, which are the user message or sequence, and then you have a special memory sequence that you also add as part of the input.

I would say parametric memory generally isn't applicable right now, and we don't spend too much time thinking about it, because it's not scalable, at least currently. You can learn something about how a company works, but that's going to change really quickly. And right now, given current compute constraints and the algorithms we have for continual learning, it's cost-prohibitive to keep retraining that model, given how fast organizations — or even people — move and change in their lives. But technically, you could also represent memories as just a vector.

Graph and tree topology

Taylor: Gotcha. So ultimately what I'm hearing is that this is a good starting place if you're beginning to think about memory — maybe trying some things out before moving to something like graph and tree topology, which seems to be a little bit more in the zeitgeist of the times and what people are starting to focus on. Could you talk more about graph and tree topology?

Devin: Yeah, exactly, that's a good point. I would say token-level sequence is basically just put it in your AGENTS.md. That's the simplest way to summarize that group of memory.

And then the second group the paper breaks out is graph and tree topology. There's a lot to this category. Graph is a bit overloaded as a term — there are different types of graph. Zep, for instance, is a known temporal graph.

Maybe taking a step back: what is a graph, and how are graphs represented? There's a spectrum in terms of graph-like properties. So you can have the extreme of a true graph, where you're actually storing entities in the graph, and those entities have well-defined relationships between them. You're going to have a GitHub issue that has a link to a PR, for example. The GitHub issue is an entity, the PR is an entity, and the relationship is "linked to" — or "closes" would be another one. It's a very strict kind of structure.

And then you have things like Zep, where you have a fourth piece to your graph — not just entity, relationship, entity, but a versioning component or a time component, where you associate that relationship with a specific point in time or specific version. That's the true graph representation.

And then there's the tree-like structure, where the main relationship you have is parent-child. You might have a topic that is, let's say, Dosu. And then Dosu has a set of products that are children, and then those products have features. So you have a very simple tree-like structure to the relationships in your graph.

This also leads pretty well into file systems, which we'll talk about later on the storage side, because a file tree is a hierarchical structure. So you can represent simple tree topologies like file systems.

Another common pattern is where you have a set of facts — as we were talking about before, something extracted from a transcript — and then they have a parent, which is a topic. So the facts are the leaf nodes, and you have higher-level nodes which are summarizations across many facts related by some topic. We see that as a common pattern; we do that within Dosu as well.

The other thing I'll call out that's sort of graph-like, and that I think has gotten a lot of attention, is LLM wikis. So Andrej Karpathy's famous LLM wiki tweet — which is maybe a hybrid between a real graph, quote unquote, and a file system. You have sets of files, they have hierarchy from a parent-tree perspective, so there are folders or concepts that have sub-concepts in higher-level concepts. But then they also link to each other.

That's where the name comes from, I think — Wikipedia, the most famous wiki. Wikipedia doesn't really have that much structure when you look at it. It's not like a doc site where you have subdirectories and you're navigating a file tree; you navigate by clicking links to other docs. And with Wikidata, there's actually a lot of structure behind Wikipedia. But at least within the purest or simplest form of an LLM wiki, your main relationship is just "related to." So you have some hierarchy, parent-child, and then you just have a relationship across different pages within the LLM wiki.

Heterogeneous composite

Taylor: Gotcha. And that makes sense to me — why a lot of people are starting to focus on this graph representation of knowledge — because two things stand out from what you said. One is that a fact or a piece of knowledge might not always be the same forever, right? If you change your name, you live in a different spot, your address changes, something like that. And then just the level of fidelity of the data: you might have an executive summary, or you might have a large corpus of information that you want to tie to a topic. So you get that better hierarchical view. Your data has a bit more structure than what we talked about with token-level sequence — just having that markdown, that's the file, and that's something that's harder to keep up to date. Do I have that right?

Devin: Yeah, I think so. I think there are degrees of fidelity and abstraction and structure, really. That's what it comes down to: how structured do you want your knowledge to be?

Taylor: It's interesting thinking about data and what taxonomies make the most sense on that front. I think that brings us to heterogeneous composite. Don't say that five times fast — it sounds like a very clinical or very intellectual way to describe it. Could you talk about heterogeneous composite? Does that just mean blending a few of these different methods together, or something else entirely?

Devin: Like I said in the last webinar, it's just a little bit of everything. It's a very fancy way of saying you just put it all together.

As we mentioned in the previous webinar, there's no memory system, at least today, that is the best solution for everything. So for certain applications, maybe your queries are really structured and you have a really well-defined topology, and graphs are going to perform the best. But in other worlds, where you have a pretty simple data model, token-level sequencing or just text files is more than enough, and actually probably more performant.

Heterogeneous composite is: how do you blend those together to have a better overall system? So some pieces of memory are maybe just text that gets dynamically loaded or injected into agent sessions, versus also having a graph component for specific queries that need structure, or more of a semantic layer where you use vectors to search across.

So usually what we see is that products like Dosu or Letta and others in the space are composites. They're trying to give you the best performance, which means combining different parts of the system for different types of tasks — so holistically it's the best overall. I think it's mostly about combining different pieces.

MemCube is kind of an interesting one. I don't know a ton about the system, but they define a MemCube as text, and then they also have parametric payloads as part of it. So it's: how do you give structure to the different types of memories explicitly in an input? Which I think is interesting.

Storage tiers: vectors, graphs, file systems, and SQLite

Taylor: That makes sense. And for those who've been in tech for a while, the "well, it depends" answer feels like this category, right? It's rare that one tool does everything well, and there are always trade-offs to consider.

And speaking of trade-offs — storage, and thinking about access as well, is worth discussion. In terms of storage tiers and some of those examples, can you share a little bit more about how we can think about actually storing this?

Devin: Yeah, the storage side is really interesting. It's also evolving constantly.

There's what the paper calls in-context registers, or in-memory. That's essentially just your AGENTS.md, CLAUDE.md, or anything else that's auto-injected per session. You're not really thinking about where it's stored — it's just in memory, so there's no search API for it. You just have it present always.

Then there are a lot of different specialized engines for searching memories. One really common one, especially when LLMs first blew up, was vector stores — RAG, GraphRAG. There's a flavor of that that's a little different, but with vector embeddings you're going to store your data in a Turbopuffer or Chroma type of embedding store, where you're going to be able to query that information based on similarity — you're trying to find similar memories based on your query.

The other side is the graph database, which we've touched on a bunch. Neo4j is probably the most popular there, but there are actually a lot of really interesting graph-related projects. PuppyGraph is another one we've looked at. There are a bunch of Postgres extensions to do graph queries on top of Postgres.

The key thing that embeddings do well — the vector side — is that you're able to retrieve things semantically and not have to worry too much about structure. Just dump things in, embed them, and then if something similar comes up in the future, you can find them. The graph side allows you to be much more descriptive in your queries and more precise about what you want. And we'll get a lot more into that in the next episode, when we talk about retrieval and search.

And then the other storage format, which is probably the most popular today, is just file systems. First everything was about RAG — embed everything, that's how agents are going to get information. Then agents got really good at bash. And it turns out, because they're fine-tuned on bash, file systems are a great way to store information. So literally just having things on disk is a great way for agents to find and search through memory systems, and there are a lot of memory systems that represent data this way.

Actually, going back to heterogeneous composite: I think we're seeing a lot of systems where you have a file system as a way for agents to do discovery, but then you have either a graph or a vector search engine, or both — the flexibility and exploratory nature of the file system, plus a structured query language.

The other thing maybe related is SQLite — actual databases. I think agents love Postgres, agents love SQLite. It's a great way to store information as well. So we've seen a lot of interesting systems where it's just a SQLite database, or just a set of Postgres tables where agents can write SQL against their memories. That's pretty reasonable as well.

Start simple: put it in the Git repo

Taylor: That makes sense as far as what we talked about within the graph section — how do we version things, how do we try to enforce or steer our agents when they're querying memory? Like, hey, use this tool, or here's where the project lives, and then pairing that with the file system. I think we have a couple of things later to talk about on that topic, maybe around Git.

But to close out this section: it seems like most teams will pick one of these solutions and try to abide by that representation of agent memory within their storage solution. Do you think that's backwards — saying "let's use Postgres" and then trying to work backwards from that? Or does it make more sense to dig into what's going to work best, talking about the problem first?

Devin: My take, like most things in engineering, is keep it simple. So I think most people start with a file system representation. Typically it's actually just put in the Git repo — that's by far the most common thing we see when people try to build anything custom in the memory and knowledge space. Just throw it into your Git repo. Agents can access it, they can use bash, so you don't have to worry about the search side. And it's simple. You're already working there on a day-to-day basis.

But over time you see that things break down, and we'll get into that much more in later webinars. You don't want friction to manage knowledge, and unfortunately, at least in current Git systems, there's a lot of human intervention that has to go into updating information. So it's not the best long-term representation, but it's a great place to start. Starting simple and then progressing is important.

To be fair, I think that's what we're also doing as an industry. We started off — and a lot of really complex memory systems took off, the Zeps, temporal graphs, GraphRAG, all these things that are really cool but very complicated. And then there was this emergence of, actually, let's just throw some files into Claude. And that works pretty well. But now we're starting to see the limitations of that. And I think we're going to climb the complexity, or sophistication, of the technology over the next year or two.

Fidelity vs abstraction: artifacts and knowledge

Taylor: Nice. Yeah, I'm excited to see what happens.

Moving into our next section — we talked a little bit on our previous episode about memory fidelity beating abstraction, and you had mentioned some examples about the meeting transcript and then "summarize this," and that seems to be a mostly lossy kind of interaction. You talked about keeping it simple, and I think many teams will start there because it is simple and repeatable. Do you think there's anything teams need to think about as they're managing their memory and building these types of systems? How can they factor in fidelity versus abstraction? How do we level-set that?

Devin: Okay, I actually think this is a really interesting topic. The paper touches on it, but I don't think it gets into the practical applications of knowledge storage.

From the paper's perspective, it shows that at the end of the day, if you want agents to know specific things, then you need them to be able to find that actual specific piece of information. If you're summarizing, and you're using LLMs to extract information and transform things, it's going to be inherently lossy.

So what this gets into — and what we've talked a little bit about, and what I think is really important — is this separation between what we like to call artifacts and knowledge.

Artifacts are not canonical. They are not to be trusted as a source of truth, but they are historical artifacts about what actually happened. Examples of artifacts in the human sense are Slack conversations and Slack messages. Or Notion pages about a design doc, where it's useful context but who knows if it was actually implemented the way it was described. Or the Linear ticket about a customer bug, which gives rationale for things — you can't take it for truth, but it's a very useful thing to reference.

And then separating that out from knowledge, which is the trusted memory, the canonical source of truth, that agents know when they see it they should trust it. Extending the database analogy, that's a materialized view of all those artifacts.

What's really important, and the paper touches on this, is that you want, one, to separate out your knowledge and your artifacts — or semantic memory and episodic memory is another way of framing this, in the more traditional human memory design. And two, you want your knowledge to be able to link back to the corresponding artifacts, or episodic memories, that contributed to it. That way you have both the summary, or higher-level semantic description, of what your agents need to know — which is easier to retrieve and actually very efficient — but you also have a historical record for queries that need to be exact, or need to understand how things changed over time.

Artifacts are not canonical. They are not to be trusted as a source of truth, but they are historical artifacts about what actually happened. Extending the database analogy, knowledge is a materialized view of all those artifacts.

Versioning and branching knowledge

Taylor: And again, borrowing the database analogy, there's a reason we don't truncate our tables after we've run a successful select on them, right? It makes sense to me why we'd have artifacts, that raw data, to be able to pull from. And then later on, if we have a different question or a different framing, being able to make sense of it through a different lens makes a lot of sense.

Devin: I like that analogy a lot — your artifacts are sort of your data warehouse, where you're doing your backups too. So you have your tables partitioned by date and you can understand, okay, what did our database look like at this point in time?

That also gets into a really important part of knowledge that comes up a lot in production but isn't talked about in the paper at all, which is that versioning is really, really important when it comes to knowledge. This is why I think Git is a really common starting place for companies — you kind of get that out of the box. You get some level of versioning, some level of history, or rewind and replay behavior. I think there are some limitations to GitHub, but Git as a principled way of thinking about versioning knowledge is really important. You need that historical archive of what's happened. I think Mesa.dev is one startup in the space doing interesting work there, around making it easy to version knowledge when you think about a knowledge file system or any sort of structure.

And then the other aspect of Git that I think is important for knowledge is branching. If you want to trust your knowledge separated from artifacts, there's a lot of work-in-progress knowledge — knowledge that is true as of your branch. You're working on a task with a coding agent, and there are a lot of changes you've made that are now true on your branch but not true for the rest of the organization.

So I think there are interesting requirements when thinking about knowledge representation and storage, where you want to be able to support this branching factor. You can say, here's our main knowledge, but here's my overlay of knowledge for my current state of the world — and then have a process to say, okay, this task is complete, or merged into main. And how do you reconcile your little world back into the broader organizational knowledge?

Writing knowledge for agents, not humans

Taylor: I really like that, focusing on branching and knowledge, because my brain is going to Marvel movies. They had a series, What If — that branching knowledge, like what if this happened versus this, and then there's a whole sequence of events that can follow from that. But such is true when it comes to feature branches and all this development that we're doing too.

Really interesting stuff. Thinking about our next section: what does this mean for artifacts built for agents? We've talked a lot, through the history of Dosu, about writing knowledge for humans and for agents. Right now we're seeing agents move so fast through different iterations of knowledge or facts. Can you talk a little bit about that? Because whether it's markdown or some kind of embedding, some format that's ready for agents — I feel like there's a lot to expand on there.

Devin: Yeah, there's a lot to this topic. Maybe it's controversial — I think it's becoming more consensus — but I believe pretty strongly that the knowledge representation for agents shouldn't be the same thing we show to humans, for a number of reasons.

When you think about humans' representation of knowledge, you think of documentation pages, so long-form pages, and then pages organized into some structure that's easy to browse, kind of like a book. Because humans, when they want to learn something, want to read basically front to back.

Agents, on the other hand, read differently. One, they don't need prose in the same way. They really just want facts, so the information can be a lot more concise than what you'd want to read as a human. I think GitHub actually just published research about this that was really cool and speaks to this point exactly. They call it their knowledge compressor, where they found you can reduce document size by approximately fifty percent — I think it was just under — and preserve the exact same retrieval performance and semantic understanding of the doc. Because there are just a lot of filler words in language. It's not meant to be efficient; that's never been the point of language. But agents just want to be efficient.

And then the other thing is that agents are really good at reading things in parallel. If you told me I have to understand something and I have to open up twelve tabs and read them in parallel because they're all related, I'd just be like, no, I don't want to do that. Versus an agent is happy to open twenty links and digest them all at the same time, in a way that for a person just isn't possible. And so that means you can have much shorter documents that all link to each other and are a lot more focused, when you represent knowledge for agents.

Agents don't need prose in the same way. They really just want facts. And agents are really good at reading things in parallel — so you can have much shorter documents that all link to each other and are a lot more focused.

What's worth storing: knowledge as a cache

Taylor: Interesting. Devin, I'd love that superpower — reading through a technical book and going, read these chapters in parallel and then bring it all to the forefront and make sense of what was important for me to know for this task. Very interesting.

At Dosu, as people are spinning up their libraries and knowledge bases and company brain, as they start to use Dosu's MCP, I'm seeing a lot more iterations across their agents versus what's done manually by people. There's still the human-in-the-loop component, but firsthand it's just wild to see how fast people are iterating with their agents. I also liked what you said about what you do as a person when you're looking through documentation — you're drawing the comparison to looking at the sidebar and the sections, versus reading everything start to finish.

I know we're just about at time, but there are a few more things worth jumping into. Where is memory useful, and where might it be a cache miss? We're seeing lots of sessions, lots of agent activity, but it's not worth storing everything, right? So how can we think about what's useful to extract from a session or a task we're working on with our agents?

Devin: It's a complicated question. We'll probably get into this a lot more in the following webinar, so I'll just give a very brief overview of my thoughts.

I think the separation between knowledge and artifacts is important here. Because if you think about artifacts, they're not changing — they're non-canonical historical pieces. They're basically free to store, because you can store them and don't have to worry about maintaining them, as long as you have links and some structure around it.

Versus when you're trying to store what is true today, there's a cost to that, because it's evolving potentially every minute at large organizations. So you have to think about what information you want to maintain. Some information you need to always maintain because it'll never show up again — it's something a human said, and this is true, and there's no other source of information for that. We call that necessary knowledge, and you need to maintain it.

But then there's a lot of knowledge, especially in the LLM wiki vein, that is functional. It's basically a cache, where you're pre-computing knowledge so you don't have to redo that research next time your agent runs. And for that, I think the cache analogy holds. You want to be storing information that is expensive to recompute or frequently accessed, so you're getting ROI on that knowledge. If you're maintaining something that's never referenced, then why are you maintaining it? Especially if it can be recomputed — or if it's constantly changing but really cheap to read one file, that doesn't make sense either.

You want to be storing information that is expensive to recompute or frequently accessed, so you're getting ROI on that knowledge. If you're maintaining something that's never referenced, then why are you maintaining it?

Standardization: OKF and DocLang

Taylor: That makes sense. Again, database analogy — there's a reason we don't keep all of our backups of all the data and all the changes across all of time, because that gets really expensive. Someone's going to start to run out of disk space, a cloud provider, one of these days, and that's not a good spot to be in. So it makes sense to have the right checkpoints, or the right type of knowledge, or the one-offs like you shared.

One of the things I think is missing slightly: we have AGENTS.md, we have the MCP spec. Is there anything that comes to mind around standardization? I know we have things like Google's Open Knowledge Format, and the DocLang team working on DocLang. I think those are two of the better examples out there, along with other projects like AGENTS.md and MCP. We'd love to get your thoughts on standardization and what makes sense as it pertains to agent memory.

Devin: Standardization in knowledge format is actually one of the more important and interesting questions in the space right now. If you think about where we want to be with knowledge — where agents can plug in and out of companies and understand how that business operates really easily — in order to do that, there needs to be standardization in the way knowledge is structured, or interacted with at the API level.

Right now we're starting to finally see some attempts to standardize knowledge, but they're all very nascent. The two main ones I'm aware of — and if anyone on this knows of other formats, please share them, because we're really interested in this space — are OKF, Open Knowledge Format, from the Google team, specifically out of GCP, and DocLang.

They approach it differently. OKF is pretty minimal, although it's growing in complexity with every version release. It's basically a markdown file and some metadata, and it gives some opinions on suggested fields for the sections in the page, and then the semantic meanings of the metadata you attach in the front matter. DocLang I'm less familiar with, but it's more opinionated and uses XML, so it can be a lot more descriptive about what's going on in the doc.

It's actually interesting to see the origins of both of these formats, which aren't necessarily related to agent memory. OKF came out of GCP's internal semantic layer type of product, and you can see there are a lot of references to SQL and table schemas and "the meaning of this keyword, you should look at these tables" — a lot of data engineering and business analyst backgrounds and concepts incorporated into it. Versus DocLang coming at it from the document processing world, where you need an agent to effectively understand an unstructured document like a PDF, and do that in a really precise way, repeatedly. So you can see why XML, and the precision XML gives, makes sense in that format.

I'm also interested in something that comes at it from the pure organizational knowledge, company brain angle, which I don't think we've seen yet. OKF is a bit closer from my perspective right now. But it's very early days in standardization, and it's something I'm personally very excited about.

Closing advice: keep it simple

Taylor: Yeah, it's going to be fun to see it evolve. I think other tools in that space, like Letta's trajectory, are very much focused on session logs, as you know — with Decant, one of our open source tools, and helping to make sense of what's going on in sessions. It feels nascent, but I'm excited to see more standards come out and create some structure out of what feels like chaos from time to time for all of us.

Well, awesome. Thank you, everybody, for joining us today. I hope you enjoyed learning a bit more about where agent memory actually lives. With that, Devin, I'd love to turn to you. Do you have any words of wisdom, or good frames of reference, or things that are top of mind that would help round out this session — things to walk away with as people go and think about where their agent memory is stored?

Devin: I would say keep it simple, stupid. That would be my words of wisdom.

If you're a company exploring how to store your knowledge, I would advise: don't go the graph route. Start with a Git representation or file system format, and then figure out where things break down and when you need better retrieval — which we'll get into next time. So start simple, and then make your way to more complex storage formats.

If you're a company exploring how to store your knowledge, I would advise: don't go the graph route. Start with a Git representation or file system format, and then figure out where things break down.

Taylor: Wonderful. Well, thank you, Devin. So you've heard us describe four ways to store agent memory, and we haven't really talked about search — so that's what's next up for us. Next episode we'll get deeper into retrieval and give you some tips and things to think about on that front. Until then, we'll see you next time. Thanks, everybody. Cheers.

Devin: Thanks, everyone.