Scala isn't as hot as it used to be. I think the rough Scala 2->3 transition, coupled with improvements in the base Java language, emergence of Kotlin + Android support, and popularity of Python in data science and data pipelines (lets just do everything in one language became popular) kind of made Scala not quite as popular as it could have been. Plus the long compile times are a pain. However it seems to have a really high coolness ratio for a language. The few jobs I do see in Scala are very cool looking. Very few boring looking jobs.
You got it. Scala had a shot being an early mover in the JVM functional programming space, but they really shot themselves in the foot with their version transition problems and tooling issues you allude to. Java is probably "good enough" for most shops now, and if you are not bound to the JVM I really don't understand why you would go with Scala today.
I haven't used Scala for quite a while now - but a while back they had a serious asshole problem with a lot of the community.
Scala’s downfall is the culture of writing libraries with the most esoteric syntax imaginable. It’s the opposite of Ruby’s “program with pleasure”—-it is “program with maximum cleverness” and “if it doesn’t look like an obscure math equation or symbol soup, it’s probably not idiomatic Scala.” Java’s designers were opposed to operator overloading due to potential for abuse, and did Scala deliver living proof/nightmare scenario of that.
Now I still want operator overloading, because I favor enabling instead of disabling approaches, and I want, for example, new numeric types to have natural syntax that blends in with built-ins, or an equality operator that works consistently. I’ll even settle for += append to strings and collections.
But even Scala standard libraries take operator use to wild extremes. Apparently it’s not enough to use + to append collections, you must use ++. Except Vec you can add with +. There’s ++= and ++=:, and /: and :: and ::: and ?^ and ?^.
Spark Scala interface introduces =!= for inequality comparison. There’s |@| from Cons library. Don’t get me started on Akka. SBT introduces <++=
Surprisingly you can’t override ==, so Spark implements ===. And specs2 testing library implements ====.
Scala is too complicated. Most scala code bases I have worked on have no enforced structure, the language allows for all sorts of unconventional programming paradigms