Too hot and expensive to run right now but a great hedge for peace of mind against $200 subscriptions shooting up to the $4000* they should cost.
*$1000? $14,000? Who knows but everything in the middle there has been claimed.
If they "should" cost 4k in the sense of marginal cost, then you will be spending more running the same at home, because your home hardware will always be less efficient.
Yeah, I guess, but it feels like there isn't really an opportunity for anyone to do that, given how competitive the market is. If Anthropic decides to demand API rates for everything (which would make my $100/month turn into a few thousand, I guess), I won't be seriously inconvenienced by switching to GPT. And, if both of the major American providers do a pricing collusion and GPT also becomes thousands of dollars a month to use, I can choose between Kimi K3 and GLM and so on. I'd rather use Opus 5 and Fable, but I'm not going to be seriously put out if I can't. We've got three or four open models to choose from that are as good as or better than Opus 4.8, which is Good Enough, and the competition isn't slowing down. We're seeing more new competitive models more frequently than even three months ago.
So, even though there are more models to run locally that can be useful for the stuff I do, it makes less sense now to do so than it did when I got it. There are more extremely cheap options, now, and it seems likely to continue to get cheaper and better and faster, while my local hardware will always be slow and hot and only gets better via software (which has roughly doubled the speed it can run stuff since I got it, but it seems like there's less room for improvement in software now, and even at twice the speed, it still kinda sucks to use local models interactively especially on the Strix Halo).