logoalt Hacker News

lrvick • today at 5:56 PM • 1 reply • view on HN

No third party can ever provide the privacy and sovereignty of being able to run locally with abliterated models. I would do it this way even if the providers were much cheaper as I value those properties a -lot-.

In practice though, I can produce about 3B tokens/month of Qwen 3.8 Flash Next which is the model I crank almost 24/7. That would cost something like 1k/mo on OpenRouter. My setup costs about $300/mo in electricity, though getting solar soon to drop that to zero.


Replies

cortesoft • today at 6:05 PM

It doesn’t really drop the cost of electricity to zero; you have to amortize the cost of the panels over their lifetime, plus the opportunity cost of the capital outlay (or interest on the loan)

➕ show 1 reply