Eventually, multi-modality might even get offloaded as workflows, which might allow models to do this at a fraction of the data and compute required.

I am adding multi-modality to https://github.com/guilt/TinyToT, and I see that dis-aggregating capabilities, very similar to how our own sensory organs work, seems to be paying off quite well.