What HFT arbitrage does is rapidly push markets with different prices into alignment with each other. If you banned HFT but kept multiple markets, the inevitable result would be markets with bigger differences in prices for longer.

The only kind of trading that really good HFT trading pushes out is other slower less efficient arbitrage traders, but why should we want more worse arbitrage traders if the result is markets being more out of sync?

What's important economically is that traders that trade based on fundamentals can do so efficiently across multiple markets. Efficient HFT arbitrage trading helps that, it doesn't hinder it.