I cut the difference in speed by half by taking the activations on the GPU. Time to sleep but will continue tomorrow.
Have you tried e.g. Mojo that can vectorize/do SIMD without having to do intrinsics everywhere?
alt Hacker News
Have you tried e.g. Mojo that can vectorize/do SIMD without having to do intrinsics everywhere?