Home Assistant [1] can be a voice assistant (both on phone and on dedicated devices). It's FOSS and can be fully local [2]. It's really gear towards home automation though, so its suitability depends on what you want to do with an assistant
[1] https://www.home-assistant.io/
[2] Although frankly to have a fast-enough and good-enough LLM you might need to use a service
I do want to setup home assistant at some point, but my main usecase for a smartphone assistant is what I mentioned. Things like setting reminders/todo list items, creating calendar events - even sending quick messages to contacts. Basically, things I can already do on my phone, but can do quicker through natural language.