Optimizations is one reason for modern languages to compile slowly, but it's certainly not the only reason. Even at -O0, C++ and Rust are much slower to compile. The culprits are more likely much more advanced type-systems and monomorphization passes for generics. Slow compile times are much more inherent to the languages (compared to Turbo Pascal, certainly).
Not even, regarding type systems, see D or Ada compilers.
Or if you prefer to travel back to 1993, here is a developer experience using C++ as if it was Lisp Machine/Smalltalk environment with incremental compilation and hot code reloading.
"Lucid Energize Demo"
https://www.youtube.com/watch?v=pQQTScuApWk
With one of the very first ideas of a LSP could be like,
https://dreamsongs.com/Files/Energize.pdf
We got into a culture that accepted slow compilers is normal, it doesn't have to be.