One of the things I have been wanting to try for a while now is something like this with a layer per MCU. I have some crazy ideas with RP2350's talking to each other with dedicated lines fed by PIO going through a combination of interpolators and dual multiply instructions.

PSRAM, Flash, and even SD cards may not have the best bandwidth individually, but they can reach quite impressive rates when you have a shitton of them running all at the same time.

The large scale dedicated hardware systems will still have the edge for performance per watt, but the low entry level and slow incline does make these things quite appealing.