The Components of Agent Memory

Not sure it's explained here but from the graph (fig. 10) it looks like wiki + search beats dreaming, and that dreaming adds only marginal gains to the wiki + search configuration.

I'm assuming search is much cheaper than dreaming (though maybe it depends on the time scale?) so that sounds pretty important.

So far I've turned off automatic memory creation in every harness I've used. Claude Code sometimes creates some anyway, and I probably delete 95% of them. The memories it creates are almost always relevant only to the specific task that it is currently working on. If it's a task I'll need to do again, then yeah, maybe that's useful. But that isn't the case most of the time, so I delete them to avoid cluttering context.

So the main improvement to memories I'd like to see is better judgment on which ones to create.

The parts about "dreaming" make me think of how agent transcripts contain a ton of information, which I think is currently under-utilized. I'll repeat advice I've seen here a few times: ingest your own transcripts into duckdb (claude code will build that pipeline in about 10 minutes) and do some data analysis on them. Some ideas if you don't know where to start: look at tool call failures to see if you can prevent common ones, check out if you have large cache misses and what causes them, look at various efficiency metrics over time (tokens per $, lines of code per tokens, etc.) with main thread vs sub-agents, etc. I started with this sort of things, and ended up following some threads to interesting insights. For example, I found indications that I tended to make Opus 5.5 / Sol 6.1 divide their work into units that were too small. They have the capacity to work on more stuff at once than older models, and I wasn't really taking advantage of that.

I have a skill called memory-prune whose sole purpose is to take an automatically created memory, assess whether it deserves to live, get stored in code comments, a rule, an AGENTS.md file in the project, the global AGENTS.md/CLAUDE.md file, or (and this is the most often picked option) not at all and deleted.

Their progress toward cerulean chart is selling me pretty hard on no memory.

The biggest thing I want to avoid with a long running agent is volatility. Anything that pushes it into one direction or the other that quickly is not desirable in the average case. At least not to me.

This article is a lot of validation for what I am building. I'll definitely be revisiting the article a few times to take in the ideas. Memory systems in AIs are extremely important and imho you can do much better than simply training the base model for memory management.

Great article -- still have some sections to read, but it's just too much to read. Emotionally, that is.

Part of me feels the pang of regret and shame seeing so many little details I researched and dreamed[1] (ha!) of prototyping ~2.5y ago expressed and validated so cleanly. Obviously A) I never did get the prototype working nor publish the underlying ideas and B) this blog post (paper? ...plog post?) is building on it at a scale and with quality I could never dream of (fig. 3 is especially striking), not to mention all the new ideas mixed in. Still... sharing cause it helps, I think, and because I expect y'all can relate; if you're old enough, it's very similar to the feeling you get when you think about how much Apple stock you could've bought in the 90s.

Anyway, more materially:

1. Pretty interesting to see Opus 5.5, Sonnet 5.5, and Opus 5 tied on the basic memory test (aforementioned fig. 3); the Opus comparison alone seems to imply either a hard plateau or that 5.5's huge improvements were accomplished by making good practices more likely, but the Sonnet detail throws that for a loop... I suppose the plateau explanation isn't contradicted, so maybe that's right (for this benchmark).

2. Fig. 4 is gonna do rounds on Reddit and Twitter, my god. Pretty stellar, intuitive depiction of the Mythos step-change that enabled true agentic engineering for the first time this summer! The highlighting is also just the right amount of facetious; love it.

3. Pretty bummed to see no followup philosophy references. We know you have them, Anthropic nerds! Do not be ashamed of the soft and slippery.

4. I am now reminded that I knew they were trying dreaming at scale, as I'm actually on the waitlist for it! And that backs up my overall takeway, actually: I can't help but feel like they're either A) vastly underestimating the amount of CogSci literature that could help them go beyond the "dreaming is when you think in your offtime" paradigm, or B) not sharing their real learnings.

Today's RSI announcement and the final sentence of this seminal plog post being a pitch for enterprise customers supports the latter, I think. That's ostensibly a good thing from a societal stability perspective, but I can't help but feel left out -- not to mention paralyzed by fear of the known unknown...

So after reading the article, I asked a Claude Code session I'm working in at the moment about its memory and whether it needs to "dream" to reorganize itself because I honestly wasn't sure what it was doing about it.

It explained that unlike Chat, each project in Claude Code creates memory files in ~/.claude/projects/[project path]/memory/ with notes about both the project and my impromptu directives or instructions, and a MEMORY.md index file that's loaded for every new session. It told me that my current project - which has been going on for a while - has 61 files in its memory, so it suggested that now in fact would be a good time to clean them up. Perfect.

It gave some examples of things things it has memorized: Apparently I once told it to always capitalize the first letter after a colon. I don't remember doing that, but hey, that's basic English grammar, dammit, so I can see myself getting pissy about it. It had saved that bit of petulance on my part in the memory.

That's great, but it was saved only in the memory folder of that project. This is good to know! I hadn't realized that stuff wasn't saved globally. Or rather, I vaguely remember turning that option off at some point and now I'm realizing that means I need to do some maintenance manually if I don't want to have to constantly remind Claude to capitalize properly, (and to always refer to me as Mr. Big.)

Definitely a good article for me to read right now.

[flagged]

They're not even using holographic projections and lower dimensional manifolds ..

Looks like a promising way to bind customers.

Shh... tiny agents is dreaming...

Such a bullshit article, as much of Anthropic "research" is these days.

> By “agent memory,” we mean knowledge that Claude gains from experience—including interactions with users and tasks it’s been assigned—and that it retains between sessions, saved as context it can read later.

That Claude rarely, if ever, actually "remembers". Since these are just random markdown files randomly injected into the tiny context.

"I'm sorry, I'll make a note of it" and will proceed to write a 15000th note with the same content it's already written dozens of times before

> We compared the memory folders from one run each of Claude Sonnet 3.7 and Claude Fable 5, when each was about 10,000 steps into the game.

What a useless experiment

> That general guidance was enough for Opus 4.7 and Opus 5 to use memory effectively

Define "effectively"

> In this case, prompting Claude to read its memory files before answering—and then to explicitly test its theories—nearly doubled its benchmark score

Then it's not "memory". It's just telling the model to read a text/spec file before implementing something

> Claude doesn’t load all of its auto memory into every session’s context window. It loads only the first 200 lines or 25 KB of MEMORY.md

So, random injection of random things into context that depend on whether the short ToC entries have any relevance on what Claude is doing, and which will be ignored.

And you have to explicitly tell the agent to read specific files otherwise it will ignore them.

What is this bloated article about?

[flagged]

Another day, another Anthropic marketing article trying to deceive people into believing Claude is conscious. Absolutely insane anybody takes this company seriously when they describe "writing notes into an .md file to be re-injected into context later" as ""dreaming"".