It's true that it's all packet-switched rather than circuit-switched, but IMS traffic is still handled differently than regular data traffic. (The most user-visible implication of this is the QCI difference: cellular phone calls are much more reliable at poor signal strength and high congestion than over-the-top VoIP calls).

That's not quite true. VoLTE was an optional part of 4G in the beginning. I had one of the gaming ASUS phones that shipped with 4G, but no VoLTE. 6 months after getting it, the US phone carriers conspired to make VoLTE mandatory for connecting to their towers, effectively bricking my device. ASUS never issued the firmware update that would have enabled the feature in the modem, which had the capability, and the community hack to enable it never worked for me.