(Inexpert ramblings follow)

Part of the problem of this is likely that the deep meanings of words you might use in chat to describe a business problem or task that you wish to see implemented are essentially inseparable from scenarios in which they are used.

Putting aside the bouba/kiki effect and anything like it, complex words only have meanings from usage. That usage is built on grammatical structures that also emerged only from usage.

(This is something I was taught as a sort of fact but I gather it was basically abbreviated Wittgenstein? … who I cannot claim to have studied)

So what you're looking for is a language model where fundamental word meanings are encoded without the weight of knowledge of where they come from. This is plainly difficult, because complex words are used by extension and analogy, and these days, many are neologisms or portmanteaus, even ephemerally — developed and discarded within a single context.

Reasoning about language itself to its full meaning is quite hard.

Like my favourite word of the moment: "obscurantist". You see that and you have a glimmer of what it might convey. But why do you? How much of that comes from explicit grammatical knowledge of suffixes, and how much from simple experience of using words like obscured, informant, attendant, dentist, artist?

So a language model might be able to deduce what "obscurantist" logically means when applied to a tract or to a person. But without lots of parameters covering its use, could it properly grasp that in some circles it would be pejorative to the point of being deeply offensive?

I think the best hope for your pluggable knowledge base idea is model delegation: strong reasoning models that know how to dictate to smaller specialist models and draw conclusions from their responses. I find myself wondering if there's any way that can be done the same way that, say, Gemma 4 12B's integrated vision encoder works — within shared weights, somehow, without them to speak in some intermediate language, like a partitioned brain. But I find it difficult to believe that is pluggable at all.