You could try ours. They're very small and english only right now (training Spanish now). But they're a CPU optimized architecture and even the largest size will work on a single thread with realtime streaming.

https://huggingface.co/collections/futo-org/asr4all

You can run them client-side using wasm-simd. I setup a space for the small sized model. There's a big step up in accuracy going to the medium size though:

https://huggingface.co/spaces/futo-org/asr4all-client-side

Parakeet. For my use it seems about equal to Whisper in WER, but is >10x faster.

I'm looking for a local model that can do diarization, have any suggestions for this?

I have https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize open in a tab to try out

A leader board on HF: https://huggingface.co/spaces/hf-audio/open_asr_leaderboard

I have been using different Parakeet models for STT over the past months and been pretty impressed with the local performance.