alt Hacker NewsHonestly, whether you think burning current SOTA to hardware is an overinvestment risk depends on what your definition of intelligence is. If you think intelligence is something that can grow like height such that 18 months from now we will basically be bowing down to machine god giants that are running on B200s, then investing in ASICs is the wrong move. However, if you you subscribe to the (very reasonable view) that intelligence is more like a round ball that we are trying to make as spherical as possible (ala Francois Chollet's writings), then at some point the ball will be smooth enough for most people and many tasks.
It takes about 18 months to go through the design, verification, and manufacturing process if you move at breakneck pace. Design could probably be sped up.
About 18 months ago the top model was GPT-4o. Not great by today's standards, but still good enough for many tasks (certainly a big chunk of chatbot queries). The current SOTA covers far more use cases, but importantly at a level that surpasses many thresholds of utility.
Replies
At a 50-100x speedup even a GPT-4o class model could perhaps compete with much newer models simply by thinking deeper, doing harness-controlled Ralph loops, etc. Sure, then it might be "only" ~2-5x faster, but, you wouldn't need to throw all the ASICs into the trash bin.
One could also imagine hybrid models, where part of the model is burned into ASICs and part of the model exists in VRAM/HBM2 so it can be updated.
I don't have enough low-level knowledge to evaluate the technical or economic feasibility of the above ideas, however.
I think part of this thinking is in reverse. What I mean by that is people are saying something like "Imagine if 18 months ago a company started to create an ASIC based on GPT-4o (or whatever was good at the time). By today they'd have a model that is not great by today's standard but good enough".
Maybe, but what is the shelf-life of that 18 month decision? Barely good enough today, when it launches, starts to get worse and worse every month going forward. You have to recuperate that investment on your depreciating asset.
And you are competing against anyone with the foresight to use a TPU instead, and the benefit from any new paper that finds how to distill, quantize or whatever better so their solution gets algorithmic boosts while you are locked in.