I have not looked at the implementation but isn't the idea to write a Lispy language in Rust (in other words, Mathematica the language) and then write the differentiation and other routines in that.
No, they're implementing all functions, all matching etc. in Rust.
They had to patch the Rust compiler to natively support AutoDiff.
Contrast with Julia where it can be a regular Julia library,