That's not the reason, Pi0 was basically predicting at 10hz and predicting a temporal chunk up to 50 points so it could go up to 500Hz.
It's slow because the original telop is slow, and the learned controllers through imitation learning is always a bit slower.
Source : i work on this (not at PI)
Another practical reason is that it's dangerous.
Pi0 uses ARX robot arm which weights 3-4kg per arm. It can easily break things or harm people if you allow it to move fast.