Rayon optimises for throughout, not latency. If you want to ensure a maximum latency cadence, use something like micropool: https://github.com/DouglasDwyer/micropool.
Here is an interesting video about it: https://youtu.be/QFQkqFSg8Z4.