logoalt Hacker News

onlyrealcuzzo • yesterday at 6:18 PM • 1 reply • view on HN

Yes -> every 18 months they've gotten 90% more efficient for the same level of quality for about 5 years. There's little sign that trend is slowing. If anything, there's reason to believe that System 1 models (plus potentially 1-2-3 workflows) may increase that over the next 3-5 years.

You'll know when the trend stops -> when the intelligence differential between smaller models like 7B starts to grow instead of shrink from 32B models -> that means 7B is getting about as smart as it can get. Then, 32B will follow next, then 70B, etc etc.

We haven't yet seen that at any size AFAIK.


Replies

thefourthchime • yesterday at 6:22 PM

Andrej Karpathy said once that he expects superintelligence could fit in 1 billion parameters.

➕ show 1 reply