alt Hacker NewsOkay here's something LLMs can't. They can't solve problems that are longer than ~10 pages of math. They also can't maintain codebases without supervision. It's because they are have no memory and use various tricks to supplant that fact.
Okay here's something LLMs can't. They can't solve problems that are longer than ~10 pages of math. They also can't maintain codebases without supervision. It's because they are have no memory and use various tricks to supplant that fact.
Now, how long before someone rolls out some sort of 10M context hybrid attention active context management monstrosity and ruins this guy's "can't"? Start the clock.
My opinion of claims like "LLMs need memory to manage codebases" has also hit the dumpster bin a while ago.
Why would knowing how to make a maintainable change to a codebase require any more "memory" than knowing how to play an optimal chess move? The codebase is the memory. A sufficiently capable LLM can ingest it, figure out what changes to make, and make them.