I agree that if you have the weights you can use/train a model with the same architecture, and that you won't get the exact weights on your own due to randomness. But isn't data an extremely important part of your ability to effectively train/finetune? It might be much harder to get close to the level of the open weight model if you don't have the data that made it, which is why I think the open weights vs open source distinction is useful.

There's more to open source than reproducibility. For instance introspection which is even more important with weights since detection of backdoors in models is NP hard IIRC.

Isn’t this a bit like saying “ ‘open object files are not open source’ is a complete misunderstanding” because “You can’t rebuild the executable from scratch, but you can’t do this even with the source code anyway (because of build nondeterminism / compiler versions / etc.)”?

> (As an aside, this is why the "open weights are not open source" thing is a complete misunderstanding. The weights themselves along with the documentation give you enough to fine tune the LLM. You can't rebuild it from scratch, but you can't do this even with the data anyway (because of randomness!))

They could give you the random seeds? (Assuming you carefully train in such a way to remove other sources of randomness, like concurrent execution.)

> open weights are not open source" thing is a complete misunderstanding

> You can't rebuild it from scratch

There's is extremely clear and misunderstanding-free.

Open weights is not open source.

> You can't rebuild it from scratch

> It's entirely reproducible from the available documentation

You have a very interesting understanding of "reproducibility", I'll give you that :)

But even with that, there are plenty of technical details (especially in regards to the training process) missing from the tech report that leads to these weights not being reproducible in any sense of that word.

the beginning of the arguments was good but the "because of randomness" is wild