alt Hacker NewsI think, may be actually wrong, that most of us do not consider running a model locally a way to save money. It is a way not to spread personal info around.
I think, may be actually wrong, that most of us do not consider running a model locally a way to save money. It is a way not to spread personal info around.
Anyone running LLMs at home will come to that realization quickly, if they’re looking at their power bills. Even feeling the heat output of a computer running at 100% in your office makes it clear.
I was responding to a lot of the comments saying this was a reasonable way to avoid paying for tokens or subscriptions. I don’t want anyone getting the wrong idea that this is a way to save money if that’s their priority.