alt Hacker NewsThe qwen models really seem to have this as a failure mode, its so annoying having a proper trace ending up in !!!!!! Garbage.
The qwen models really seem to have this as a failure mode, its so annoying having a proper trace ending up in !!!!!! Garbage.
Wait in a regular sentence, what is the probability of "!!!" being followed by "!"?
Sounds like the model is not following a proper probabilistic choice here, so maybe more a programming error than a model training error.