alt Hacker NewsPrivacy is a great reason, but independence is another. It’s very nice knowing that you’re going to get the same reliable product every time you call the model. Nothing is going to change unless you decide to change it.
Privacy is a great reason, but independence is another. It’s very nice knowing that you’re going to get the same reliable product every time you call the model. Nothing is going to change unless you decide to change it.
That is not a counterargument to cloud inference though. You can also run open weight models in the cloud, and it's still cheaper. So privacy really is the only motivation to run on local hardware.