alt Hacker NewsIs this admission that google’s proprietary chips etc. are not cutting it? Why would you need a bunch of nvidia GPUs if you have your own silicon? (AFAIK they have their own for both inference and training do they not?)
Replies
Kinda; while it does show that overall Google's proprietary chips etc. are not cutting it, it doesn't say if the problem is the hardware itself or the factories to make more of the hardware. Without more information, it could be that Google's hardware is 100x the energy efficiency per token, but they can only make enough hardware for 1% of the tokens there's currently demand for: 1% of your product being 0.01% of your costs isn't nothing, but it leaves the other 99% at full price.
Supply and demand? Bubblists seem to think there's an infinite supply of chips, power, and water to make as many chat bots as possible; physics, as usual, dictates limits.
Not necessarily, just that they don't have as many as they can make use of, and that xAI can't make more valuable use of them than renting them out.
Yes, it is issue of scale, google had to restrict usage because hardware are not available, regardless of what kind of hardware that is
Or it’s paying to make sure competition can’t buy said compute. Also isn’t Google an investor in SpaceX anyways?
It could be, or simply we are so far away into chip shortage that even google needs to buy from other people’s pot.
At this point you are not buying a particular chip. You are buying whatever compute you can get.
Its because none of the promised Data Center and NVIDIA hardware deployments described in NVDA earnings calls have actually happened. Once more Ed Zitron has the goods: https://youtu.be/zbKDmkJPVvI?t=482
Or alternatively there is simply a huge demand for compute and this is helping them fill a short-term need. Keep in mind if you saw in the article there is a 90-day cancellation clause. This is a nothing burger.
How do you come to this conclusion? All it means is that spacex has compute and google does not.
Suppose tpus were theoretically a million times better, but cannot be produced due to supply chain constraints, this action would still be rational.
My personal take is that this really shows how bottlenecked the entire supply chain is. For such an important commodity there are shockingly few players ready for scale.