logoalt Hacker News

teaearlgraycold11/08/20240 repliesview on HN

The main issue is the models you need to do the job are too big for most consumers, even people with nice video cards. You'll need a couple L40S GPUs at minimum. Maybe a few H100s more realistically.

So do you want to buy tens of thousands of dollars in GPUs or do you want to rent them second-by-second? Most people will choose the latter. I understand you don't trust the infrastructure and that's reasonable. If self-hosting was viable it would be more popular.