logoalt Hacker News

xnxtoday at 2:06 PM2 repliesview on HN

It's interesting to compare and contrast LLMs and biological brains. Both use a form of "compaction".


Replies

un-diletantetoday at 3:13 PM

Compaction is not even an intrinsic function of an LLM, but of it's harness or whatever code is calling it. When an LLM compacts something it's just creating a summary of the previous conversation using the same mechanism that it uses to generate any other text. It doesn't "forget" the text from before the summary, it just no longer receives it when it is invoked.

In contrast, when we forget something we are not summarizing anything, we are just losing the ability to recall something. We might remember what we have forgotten later on, showing that it is still somewhere in the mind, and that the reason we have forgotten it is not necessarily because it has been discarded.

Even then, what this article is talking about is not even normal forgetting as a function of the brain. It's talking about a degenerative and dysfunctional form of memory loss where different memories are confused with each other.

voidhorsetoday at 2:08 PM

That's largely because we intentionally model llm "brains" after the human brain.