When you barely need said device, or you occasionally need many of said device. Not to mention you're renting more than just a GPU from them, you're getting the rest of the server too.
I rent from runpod a handful of days a year (maybe a week or so total?) and I've estimate that I've spent at most $300 total in the last 2 years.
- I get a 3090 or 4090, sometimes a 5090 or a pro GPU with 40+gb vram.
(I have no dGPU).
- Usually they have >100gb ram (my new PC has 64gb, the old one had 32gb)
- 20-30 cores available (my new PC has 20, old was 4gb ddr2(!))
- 1-2tb of disposable nvme storage (my old PC didn't have nvme at all, my current PC has ~800gb free of 2tb)
- fast network. I don't always test it, but I'm usually done downloading ~700gb of models in under 2 hours; locally this would take around 15 hours.
- electricity is far cheaper in random east European/Asian City than it is in Los Angeles. Some GPUs cost less to rent than they do to run locally, never mind the cost of buying the GPU itself.
Paying runpod or vast or just about anyone else is _far_ cheaper than buying this hardware myself.
alt Hacker News
When you barely need said device, or you occasionally need many of said device. Not to mention you're renting more than just a GPU from them, you're getting the rest of the server too. I rent from runpod a handful of days a year (maybe a week or so total?) and I've estimate that I've spent at most $300 total in the last 2 years. - I get a 3090 or 4090, sometimes a 5090 or a pro GPU with 40+gb vram. (I have no dGPU). - Usually they have >100gb ram (my new PC has 64gb, the old one had 32gb) - 20-30 cores available (my new PC has 20, old was 4gb ddr2(!)) - 1-2tb of disposable nvme storage (my old PC didn't have nvme at all, my current PC has ~800gb free of 2tb) - fast network. I don't always test it, but I'm usually done downloading ~700gb of models in under 2 hours; locally this would take around 15 hours. - electricity is far cheaper in random east European/Asian City than it is in Los Angeles. Some GPUs cost less to rent than they do to run locally, never mind the cost of buying the GPU itself.
Paying runpod or vast or just about anyone else is _far_ cheaper than buying this hardware myself.