alt Hacker NewsMaybe that makes sense if the company already bought the hardware.
But I currently think the fundamental reason the cloud models often make more sense is based in how the models work. My current understanding is that once these models are loaded into GPU memory, the model can respond to multiple prompts in parallel. So a cloud vendor that receives lots of user prompts at the same time can achieve really good hardware utilization. These cloud vendors can then spread the cost of the hardware over many more users than you can with a local model, leading to lower prices per prompt in the cloud.
If you are an individual, almost certainly, unless you are using LLMs very heavily.
If you are a business employing many people all of who use LLMs reasonably heavily? You can still spread the hardware cost over many users and respond to multiple prompts in parallel. At some point buying the hardware will make sense.
I would not get a dedicated server to run a single small VPS. On the other hand if I needed a lot of capacity a dedi is cheaper than a big VPS (or lots of small ones).
There is also a benefit for anyone who has confidential or secret information.