> LLM's power comes with it being a monolith.

Does it? All of the latest frontier models that have made technical details publicly available are using MoE; smaller specialized units that work together.