Very slow currently, I added the benchmarks in the README. To go faster it needs to implement inference faster than the current float32-only kernels.

The Python libraries are themselves written in C/C++, so what this does performance-wise is, at best, cutting through some glue. Don't think about this as a performance-driven implementation.

PyTorch MPS is about 10x faster per the README.md.