My next project will be released under a GPL-like license with exactly this condition added. If you train a model on this code, the model must be open source & open weights
Not sure why the FSF or any other organization hasn't released a license like this years ago already.
In light of the fact that the courts have found training an AI model to be fair use under US copyright law, it seems unlikely this condition will have any actual relevance to anyone. You're probably going to need to not publicly distribute your software at all, and make such a condition a term of the initial sale. Even there, it's probably going to be a long haul to get that to stick.