But even given that, how is it that everything doesn't work very well?

The normal functioning of markets would be that badly-working things are slowly driven out, while well-working things grow and replace them. Even without any reference to financial markets, this is simply what you expect to happen when people have a variety of things to choose from.

I could hypothesize that markets have evolved to the point where it's impossible for new things to grow unless they are already shit. Perhaps because everyone's too busy working for the shit things (which is partly because the government keeps printing money to the previously successful things in order to prevent the economy collapsing and therefore landlords got to charge exorbitant rent) or perhaps because they just don't have any money because of the above, and can only afford the cheap shit things (but a lot of the shit things are expensive?) or perhaps because people are afraid to start new things because they're afraid of the government (I've observed that not infrequently on HN, also something something testosterone microplastics) or perhaps because advertising effectiveness has reached the point where new things never become discoverable and stay crowded out as old things ramp up advertisement to compensate or perhaps we're just all depressed (because of the housing market probably).