logoalt Hacker News

toxikyesterday at 5:44 PM3 repliesview on HN

$10k is well outside my budget for frivolous computer purchases.


Replies

zozbot234yesterday at 9:34 PM

It would be plenty in-budget if the software part of local AI was a bit more full-featured than it is at present. I want stuff like SSD offload for cold expert weights and/or for saved/cached KV-context, dynamic context sizing, NPU use for prefill, distributed inference over the network, etc. etc. to all be things that just work for most users, without them having to set anything up in an overly error-prone way. The system should not just explode when someone tries to run something slightly larger; it should undergo graceful degradation and let them figure out where the reasonable limits are.

bdangubicyesterday at 6:06 PM

99.97% of HN users are nodding… :)

show 1 reply
stefsyesterday at 9:22 PM

yeah, but if you really really wanted to and/or your livelyhood depended on it, you probably could afford it.