logoalt Hacker News

dghlsakjg • today at 5:58 PM • 3 replies • view on HN

How does this make you less reliant on third parties?

It seems like it just shifts which third parties you rely on. I think local inference is cool and your setup sounds dope, but you still have third party hardware manufacturers, and third party models in the mix.

What is the advantage of this over OpenRouter, which lets me select for, jurisdiction, quantization, ZDR, etc.

note: My use case can't even come close to having this make sense financially, but I understand if yours does. Either way, that doesn't make the third party argument better.


Replies

runtime_terror • today at 6:09 PM

Not the OP but I would assume: - no data retention policies to worry about in the first place - no monthly billing outside of power bill and capex - no jurisdiction issues - full control over your data - no network round trips or even worry about network outages in the first place

➕ show 1 reply
almondfestival • today at 6:48 PM

it's the difference between a one-time transaction with a third party and an ongoing, perpetual third-party liability. you buy your inference hardware, you download your model, and if you wanted/needed to, you could run inference in perpetuity (or until your hardware broke down). compare that to anthropic/openai/etc being able to throttle intelligence, deprecate models, or raise prices on you whenever they felt like it.

sure, in a very general sense you're dealing with third parties in both scenarios, but if you actually own the hardware and model weights, you can control your own destiny for the lifespan of that hardware.

drdexebtjl • today at 6:10 PM

Is this some sort of gotcha? You didn’t build your computer from scratch with sand, sticks and pebbles, so you’re just as reliant on third parties?

If you own your own hardware, you don’t have to depend on and trust third parties continually, just once when you bought it.

➕ show 1 reply