I guess the question is how much of the coding skill gap between qwen3.6-27b and gemma4-31b is down to the model architecture, and how much is because of the training data.

Assuming you can tweak the training data, regenerate qwen3.6, and get a better coder, then presumably you could have variants - e.g. qwen3.6-swift-27b and qwen3.6-python-27b. Or maybe all coding is too intertwined and you can only get splits like qwen3.6-research-27b and qwen3.6-coding-27b. Which isn't quite my pluggable-models dream, but it's a step closer.

But maybe the difference isn't the training data, it's the architecture, in which case pluggable models is probably not possible.