logoalt Hacker News

ACCount39 • today at 3:12 PM • 2 replies • view on HN

Nah. The "huge amounts of data" humans receive are incredibly self-redundant.

Incorporating non-text data into AI training runs has repeatedly demonstrated that for general reasoning capabilities, no data type beats text in value per byte.


Replies

theK • today at 3:25 PM

This is only true on the surface. Yes, making a model a good debator can be acchieved like this, but every current frontier model exhibits the same semantc confusion when it comes to actual mechanics of physical objects. Understanding how not to pass the ball through the glass is not a property of textual reasoning. This is also very visible in model generated code where the ball and the glass plane very often intersect. Easy to identify in syntax (compilers will do the job for you), easy to miss on the concept level.

➕ show 1 reply
derac • today at 3:13 PM

It is very redundant, but there is an incredible amount of it and we can "train" continuously and much more efficiently.