If weights are the source for models then ELF binaries are the source for software.

I want to caution against this line of thinking. NSA's fast16 program silently altered data during nuclear simulations, and that is also possible within open weight models. There is no reason to think there is anything like that currently, it also cannot be dismissed. Open source would include the training data so you could create a similarly capable model.

The weights are the output artifact, the training corpus and system are the source.