You're thinking only about the final step where we're just doing a bunch of matrix computation. The real work Python does in the ML world is automatic differentiation.

Python has multiple excellent options for this: JAX, Pytorch, Tensorflow, autograd, etc. Each of these libraries excels for different use cases.

I also believe these are cases where Python the language is part of the reason these libraries exist (whereas, to your point, for the matrix operations pretty much any language could implement these C wrappers). Python does make it easy to perform meta-programming and is very flexible when you need to manipulate the language itself.