The problem is that training a free and open source model costs just as much as training a closed one, but has even fewer potential avenues for recouping that investment. The money still has to come from somewhere.

I'm not sure if open weights are immune to being compromised by ads anyway, they can't serve pay-per-impression ads on the output side, but there's nothing stopping the creator from accepting funding in exchange for biasing the training one way or another.

Coming soon: Foobar-600B, a new SOTA open weight model kindly sponsored by Coca Cola, Exxon Mobil and the Heritage Foundation. Please pay no attention to the men behind the curtain.

What is a free model worth if it’s running on another company’s server farm, trained with data you do not have access to?

I agree, but what about the training data that goes into it (intentional poisoning of the training data, for a variety of reasons, $, power, etc.)

I’m wondering how long it will be until they are also “sponsored” to have ad content trained in. I personally despise advertising but nobody is building these things out of the goodness of their heart. There needs to be some ongoing incentive to train and release open models.

Similarly, I’m wondering when huggingface is going to need to start showing returns and starts putting ads into transformers etc.

To run your own chatgpt level model would require half a million bucks in infrastructure.