logoalt Hacker News

Symmetry • today at 2:07 PM • 1 reply • view on HN

An LLM is trained on the order of 10^14 bytes of text.

At roughly 1 MB/s of data to the brain (vision dominates even though the data along the optic nerve is way less than arrives at the retina) times the seconds in 20 years that's roughly 6 * 10^14 bytes. Not too large a difference.


Replies

AnimalMuppet • today at 2:50 PM

But it's different data. The human training bandwidth is dominated by physical things. There's a thing I see, and I move my hand and I touch it, and I hear the sound of the impact, and I feel the touch. And I learn how all those are correlated. I learn to build a mental map of the physical world that corresponds pretty well to the actual physical world, so that I can survive living in it.

LLMs are trained primarily on text. That trains them on relationships between words more so than correlation to the physical world.