> safe Rust is actually more memory safe than Java, since it guards against data races

No, it isn't. First of all, data races in Java are memory-safe, as guaranteed by the Java memory model. Second, even if that were safer in some sense it is only by dint of restricting the capabilities of the safe subset further, thus requiring more unsafe code, not less (e.g. because of that data race restriction in Rust, benign races, which are rather common in concurrent code and are not only safe but also deterministically correct, also require unsafe code).

The major challenge with safety is how to offer it without complicating the language much and not restricting the safe subset. It becomes much easier if you do one or both, and Rust compromises heavily on both, which comes at a price (correctness may suffer if the language is complicated, and safety and/or performance suffer when the safe subset is restricted). Some may find that tradeoff attractive, but it should be clear why others don't.

> Also note that Java has unsafe, but doesn't have the culture of plainly stating safety invariants like Rust. The unsafe features of Java are less widely used, but when they are you rarely know if a Java library has unsafe internals for performance, and if they do, it may be hard to audit

Quite the opposite. First, Unsafe is being removed altogether (https://openjdk.org/jeps/498), and overall, the remaining unsafe operations are now restricted to a very narrow, well-defined set of APIs (basically FFM, the FFI API). Second, Java's integrity guarantees (you can think about integrity as generalised memory safety) - as well as requesting exceptions to them (e.g. permissions to use FFM) - are handled in a more centralised, well-integrated, and auditable way than in any other well-known language: https://openjdk.org/jeps/8305968