alt Hacker NewsStop trying to run them locally, folks.
You don't own your fiber connection. So why try to own another rapidly depreciating, expensive, and annoying asset?
Rent cloud GPUs!
You get to participate in the ownership, data control, price control, and hacking culture without having to Frankenstein some hobbyist box that costs a ton, is distilled down to functional uselessness, and is a PITA to maintain.
Replies
If I'm gonna rent cloud GPUs I might as well just use a subsidized cloud agent like Claude or Codex. As for depreciation, that is true, but the bet is that models get better for a certain parameter count faster than your hardware becomes obsolete, such as Gemma models for example at the same 30 billion parameter count being much better than some years ago.
> You don't own your fiber connection. So why try to own another rapidly depreciating, expensive, and annoying asset?
Like a car? Because I don’t want to depend on uber or a taxi service.
What does fiber have to do with anything? I don’t need the internet to run my local models.
> Stop trying to run them locally, folks.
"Locally" is a relative qualifier if one defines locality as not being reliant on a SaaS vendor. IOW, locality does not necessarily imply execution on machines specifically owned/operated by an organization.
> Rent cloud GPUs!
This would qualify as "locally" in the above definition. There is also a case to be made that h/w ownership (GPUs included) and operation can result in a net cost reduction for some use-cases.
However, where exposing intellectual property results in regulatory violations and/or undue legal exposure, running models "locally" is not only a good option, it is the only option.