I'm not personally a fan of Java, but if I was implementing a compiler, I'd pick a language with GC. There's pretty much no downside to a GC in that context, and it gives you more flexibility when working with graph data structures.

If 'building a programming language' means writing an interpreter or VM, then I can see the attraction of Rust for that case. But writing interpreters and VMs is like 0.0001% of the programming that gets done in the world.