alt Hacker NewsThis subsidized inference is just a marketing ploy to increase prices and profit.
If common people can have a DIY setup with an open source model cheaper than those behemoths with a scale advantage, it's clear that we have been played.
Time to either self host a Chinese open source model or to just pay the cheap Chinese providers.
Yeah, local is clearly the future. Even beyond the cheap Chinese models you can install the apfel[1] stuff if you're on a mac and want a quick available onboard cli option. And I'm sure people will adapt the Flash-MoE[2] integration to be even better soon as well.
[1] https://apfel.franzai.com/ [2] https://github.com/danveloper/flash-moe