alt Hacker News> Or are AIs fundamentally different, and if so, why?
Literally: the context window.
With the human you have a window that possibly extends up to _years_. With your language model you have maybe a few megabytes which is always preceded by instructions from the model maker.
Replies
cortesoft • yesterday at 9:03 PM
Is this true, though? AIs already interact with way more data than their context window, they can make queries to external data sources, and they can make context references to external data that can be pulled in as needed.
The context window is the working memory, which is not ‘years’ for humans either.
➕ show 1 reply
That's a misconception. Context window of humans is measured in seconds and limited to single-digit amount of entities. Human attention is truly ephemeral, with a ridiculously short span. What you're talking about cannot be emulated by putting everything into the context, you need models with deeper cognitive capabilities, and the longer your context the smarter it needs to be. It can be done by increasing the depth and the size of the model, and by training it better. Test-time scaling does work (longer reasoning, agentic compression, ICL, etc), but hits the wall rather quickly if not assisted by better cognition.