alt Hacker News>Want it to go away, almost like magic? Local inference.
Ah yes, magic that costs the same as a new car.
Replies
ux266478 • today at 3:24 PM
A good setup will cost you the same as a decent house in a major metropolitan area. And then the price of a used car to upgrade your electrical to handle that kind of load, and get the cooling setup you need.
DiabloD3 • today at 5:10 PM
Tell me where I can buy a car for $1k or $2k.
Not necessarily!
If you have a semi-recent MacBook with even 32GB, you can run 20GB models that are pretty damn smart, with room to spare for the rest of your toolchain.
If you’re reasonably connected to the code you’re writing and prompting the AI at the level of the code, not the level of the feature, you can get some fantastic results.
Sure, it’s not the completely automated dreamland that’s been sold, but it’s still a speed-up on par from going from assembler to a higher-level language, which is still immense.
And for effectively free, if you have a machine that would otherwise have been considered “developer-grade” for a lot of tasks anyway.