alt Hacker NewsAt this point though gaming GPUs are not exactly affordable. You need a 32GB GPU to be able to run something genuinely useful for coding, and that's thousands of dollars. Still, happy to see open models make lots of progress. I just hope hardware prices come back to earth.
Replies
Scrappy builds aren't "affordable" but are more so than running Nvidia + optimized builds. You could put together a machine for $5000 or less that competes with frontier models from less than a year ago, albeit a little slower.
And cards like the $1600 (sheesh) R9700 have actually gotten more capable through community efforts at optimization, since their launch.
I think if the bubble can ever deflate/pop, and GPUs and RAM back off a bit, we'll see the dam burst for local AI.
Right now it's only accessible to people who can justify the costs.
If you had been paying any attention at all over the last year, there were 32GB AMD cards at like $1200-1400 and Intel at a little over $1000. I actually hadn't yet noticed they've gone up in price and had last checked only a couple months ago. But even at like $2000, that's still comparable to the nominal price of a mid 90s consumer PC for actual magic.
24 GB 7900 XTX user here. You can do a ton with less than 32 GB. It definitely drops off quick, but 32 GB is not the floor for useful AI.