I'd expect that kind of thing to be the primary use case, tbh. Maybe even running whisper models?

If it could run whisper, it'd be a solid addition to a pi based home assistant setup.

Yes. The Hailo chips are mainly for AI vision models. This is the first time I have seen them pushed for LLM. They are very finicky and difficult to setup outside of the examples. Documentation is inconsistent and the models have to be converted to a different format to run. It is possible to run a custom yolo8 model, but is challenging.