I think having massive amounts of high-bandwidth memory on consumer grade hardware could become a reality via flash.

How Flash in SSDs works is you have tens to hundreds of dies stacked on top of each other in the same package, and their outputs are multiplexed so that only one of them can talk at the same time.

We do it like this because we still can get 1-2 GB/s out of a chip this way, and having the ability to read hundreds of times faster is not justified for storage use.

But if we connected these chips to high speed transcievers, we could get out all the 100s of GBs of bandwidth at the same time.

I'm probably oversimplifying things, and it's not that simple IRL, but I'm sure people are already working on this (I didn't come up with the idea), and it might end up working out and turn into a commercial product.