logoalt Hacker News

troupo • today at 8:27 AM • 2 replies • view on HN

Such a bullshit article, as much of Anthropic "research" is these days.

> By “agent memory,” we mean knowledge that Claude gains from experience—including interactions with users and tasks it’s been assigned—and that it retains between sessions, saved as context it can read later.

That Claude rarely, if ever, actually "remembers". Since these are just random markdown files randomly injected into the tiny context.

"I'm sorry, I'll make a note of it" and will proceed to write a 15000th note with the same content it's already written dozens of times before

> We compared the memory folders from one run each of Claude Sonnet 3.7 and Claude Fable 5, when each was about 10,000 steps into the game.

What a useless experiment

> That general guidance was enough for Opus 4.7 and Opus 5 to use memory effectively

Define "effectively"

> In this case, prompting Claude to read its memory files before answering—and then to explicitly test its theories—nearly doubled its benchmark score

Then it's not "memory". It's just telling the model to read a text/spec file before implementing something

> Claude doesn’t load all of its auto memory into every session’s context window. It loads only the first 200 lines or 25 KB of MEMORY.md

So, random injection of random things into context that depend on whether the short ToC entries have any relevance on what Claude is doing, and which will be ignored.

And you have to explicitly tell the agent to read specific files otherwise it will ignore them.

What is this bloated article about?

Replies

ashishact • today at 9:57 AM

Exactly my experience with claude-code. It keeps apologising for not using the memory and keep storing the same memory again and again.

bbor • today at 11:55 AM

Reading such a passionate take as someone who has looked up is a strange experience, I must admit. Kinda like how the Bernard twist in S01 of Westworld felt. I'll try to respond as substantively as I can, regardless:

  That Claude rarely, if ever, actually "remembers". Since these are just random markdown files randomly injected into the tiny context.
Complaining about your anecdotal experiences with Claude(Code, hopefully?) isn't a serious criticism, the markdown files are optimized to not be random, and calling 1,000,000 tokens of context "tiny" seems like a plainly unreasonable bar. How much of War and Peace can you hold in your head at once and quote flawlessly, and is it more than 133%?

  What a useless experiment
The experiment is testing their fixes to the problem you lamented just one sentence prior.

  Define "effectively"
They do exactly that in figure 3, as mentioned at the end of the sentence you're quoting. Opus 5 used that advice to raise its benchmark score by 46.8%, and Opus 4.7 used it for a gain of 82.7%. The specific benchmark looks far from trivial, too: https://github.com/pgasawa/continual-learning-bench/tree/mai...

  Then it's not "memory". It's just telling the model to read a text/spec file before implementing something
It fits the colloquial definition, and there is no specific scientific definition in CogSci, CogPsych, Neuro, CogNeuro, NeuroPsych, nor philosophy. There's lots of useful ones offered in different contexts, but they differ considerably.[1] Regardless, it's well accepted industry jargon at this point: it doesn't matter in the same way that it doesn't matter than databases can never truly be "queried" because they can't engage in linguistic discourse.

  So, random injection of random things into context that depend on whether the short ToC entries have any relevance on what Claude is doing
Isn't this inevitably, undeniably true of any possible memory system, just from, say, a big-O perspective? If you have to persist every item in a list to decide which one is relevant enough to load into the same pool of memory, there's no point in the second step anymore.

  What is this bloated article about?
This article is very plausibly wrong, misguided, or even philosophically bankrupt, but I don't see any support for the claim that its topic is unclear. It's discussing the efficacy of proactive state management during idle periods for longrunning LLM-backed agents.

---

[1]: For example, did you know that modern CogSci is generally convinced that memories are illusions that the brain creates through a process that merely consults ineffable underlying representations (usually), and not an object retrieved from storage? That definition would also contradict Anthropic's wording here, but only in a way that weakens your criticism rather than supports it.

➕ show 1 reply