This is a really interesting take, I've been thinking very similarly.

I think there's a tendency to think of 'free markets' as having a lot more scope than they do - almost no countries allow free markets to determine the flow of people and restrict who can enter the country in some way. Similarly, nobody suggests the free market is a good way to regulate child labor.[0]

Especially in the US, there's a been a rapid erosion of other power structures like democracy, law, unions, etc that might serve as checks or balances against raw material interest. I think 'capitalism' often gets misclassified as 'pure competition', but it's probably better thought of as 'pure competition through regulated trade'. Pure competition, with no regulations to define a market, sounds more to me like what predated the industrial revolution (something more along the lines of feudalism)

[0] I took both these example from Ha-Joon Chang's Economics a Users Guide.