alt Hacker NewsI don’t think it will ever make sense; you can buy so much cloud based usage for this type of price.
From my perspective, the biggest problem is that I am just not going to be using it 24/7. Which means I’m not getting nearly as much value out of it as the cloud based vendors do from their hardware.
Last but not least, if I want to run queries against open source models, I prefer to use a provider like Groq or Cerebras as it’s extremely convenient to have the query results nearly instantly.
Replies
lordswork • last Sunday at 3:02 PM
As long as you're willing to wait up to an hour for your GPU to get scheduled when you do want to use it.
➕ show 1 reply
givinguflac • last Sunday at 2:41 PM
I think you’re missing the whole point, which is not using cloud compute.
➕ show 1 reply
my issue is once you have it in your workflow I'd be pretty latency sensitive. imagine those record-it-all apps working well. eventually you'd become pretty reliant on it. I don't want to necessarily be at the whims of the cloud