For command search I've replaced regular search and fzf with atuin. Works especially well with multiple hosts.. https://atuin.sh/

EDIT: The OP fails to mention zsh profiling: zprof. I discovered that atuin is my biggest waster at the moment with like 20ms and rest of the stuff I could clean up.