alt Hacker NewsCohere has a lot more to gain from pushing this narrative than anybody else. Looks like they're moving towards a policy maker rather than an innovation lab these days, which is quite sad.
Replies
I dunno — they have made a claim and backed it up?
This is a risk with any model (even in situations like this where it's still more likely to be a training error than malignant).
It just tells you that you have to think as carefully about open weights models as you would third party libraries. It's just another supply chain risk, only a bit more probabilistic.
I have enjoyed using Qwen but I do think it's worth bearing in mind that this is at least possible and it might not always be a screwup.
And that's putting aside the risk of all of this happening for more generalised reasons like the training set or RAG results being poisoned.
In practice it's surely most likely that the issue is that this pattern is in many of their own apps and they are doing RL training on their own code.
What about the claim itself being made?
The article refers to an acquisition they did in this space.
There is no reason this needs to be a he-said-she-said thing. The model is open weights. Cohere should immediately publish their code to deterministically reproduce this on a given checkpoint of this model. Then anyone can verify the claim for themselves.