Arduino offered tiny, inexpensive, easy-to-program computers and dominated the hobby space at first. This lasted for a few years, but then they started getting competition. The ESP8266 offered comparable performance at a fraction of the price, while the Raspberry Pi was about the same price as an Arduino but way better performance. Hard to compete when other companies are selling better hardware for a lower price.
For a long time I did a lot of Arduino stuff because you could get Nano clones for less than 1 EUR - which pretty much makes them throwaway, if you mess up.
I recently was doing a few projects with the Arduino Every, which is a nice board - but it's just too expensive. I did fry a few - so now I'm just using them as development board (the additional UARTs help a lot there), and for the actual project still use Nanos where I no longer care about the serial debug output, and therefore am fine with just the one serial port.
Am I only the only one who finds ESP better in almost every way? Once I discovered ESP8266 and 32, I basically haven't touched an Arduino board
It's still one of the best boards if you want to do stuff on "bare metal". While I agree they missed a few product innovations, it's still a product that is in demand and gets used by industry for real products.
> while the Raspberry Pi was about the same price as an Arduino but way better performance
If you are cross shopping a full single board computer (Pi) with a microcontroller (arduino/esp32 etc) for a project, it's almost always a sign you don't know what you are doing. With the exception of the recent Pi Pico, non of the raspberry pis are Arduino/microcontroller competitors - they are typically full blown linux computers with all the benefits and drawbacks that provides.
While you can absolutely solve microcontroller-style problems with full blown computers, it's rarely the best/cheapest option.