alt Hacker NewsIt's a few hundred bucks per month for now, but that's not going to last. At some point, the industry is going to pivot towards tracking token-based productivity because it's not going to be cheap forever unless FOSS models catch up.
Replies
zozbot234 • today at 7:01 PM
FOSS models have effectively caught up wrt. scale, see e.g. the latest DeepSeek V4 series - but they still require major hardware resources (hundreds of gigabytes of RAM for a very lean deployment targeting single- or few-users inference) to run at acceptable throughput.
Please don't call open weight models FOSS models - that's actually very wrong, unless you actually have all the training data and can modify the data and training methodology to retrain the model yourself.