>A good reality check is: if a stranger asks you about a specific part of your toy project, would you be able to explain it?

I like that as a benchmark. I think it also works with LLMs too. I have had best results with AI generation of code when I pass that check. When using a LLM, the chatbot is the stranger.

In fact much of the frustration I have with using AIs seems to be for models tuned to the situation where the user does not have the ability to explain it specifically and it has to make broad assumptions. This may be the best result for a lowest common denominator, but it's frustrating when it provides a generic solution to a specific problem.

I can see specialised models diverging to focus on different levels of assumed knowledge and expectations. Perhaps to have modes of thinking, maybe as a more tone based MOE. Is anyone doing mixture of finetunes?