alt Hacker NewsThe models are not open source. They are deeply proprietary since we have no access to the source materials and cannot reproduce the model independently. They are opaque binary blobs that the Chinese labs are just allowing other providers to run directly instead of only access through an API.
Replies
> we have no access to the source materials and cannot reproduce the model independently.
Given the USA companies have been loudly claiming the Chinese models are distillations of their models, also claiming "no access to source materials" seems dubious. As it was dubious anyway with because the Chinese publish lots of papers on how their models are designed, I'm left feeling I'm looking at the south end of a north bound bull.
Which is why we need the ability to train our own models. Maybe it will be viable to do it in a distributed computing setup one day. Research's already being done in that direction.
They are more than opaque blobs, some examples:
- One can load them up in a model explorer to see the layers and other components, how it is designed
- One can fine tune the models, which requires adding LoRA to the model and then running some training iterations
Do you expect any of the labs to have an accompanying data dump with: here’s every book ever written, newspaper article, song lyric, Disney movie, GitHub repo, etc. Oh, and we obviously never paid for any of this.
Even if you did, I doubt training is bit-for-bit reproducible, so you will always have to take someone’s word for the final artifact.