I've had some experience with audio on the web, back in the day I built a weird collab music streaming system in the browser and presented at NIME. Back then AudioWorklets were new and I didn't understand WebRTC. The initial design used Raspberry Pis and IceCast. However, the setup was not only difficult to me but beyond the abilities of my target audience. I was more familiar with web dev and figured why can't something similar work for the browser, truly naive. After lots of learning through trial and error and understanding how these types of streaming worked, I ended up with something in hindsight worked pretty simply and was beautifully naive.
It took audio input, fed it into a wasm opus encoder, output audio in "segments" of audio, and uploaded via typical HTTP. No WebSockets or WebRTC or QUIC (I don't even think I used HTTP2). The only reason it wasn't just a full encoded recording was because I wanted to make it continuous and streamed. I basically learned this via HLS, but also didn't understand HLS nor was it easy to encode and stream data (in my mental model) from the browser to the server.
On the server and streaming to the client, it just collected the segments and told clients when they were ready, via polling, similar to HLS. The client would download, wasm decode, and add the audio to a buffer. The thing is that because of the nature of the project it didn't really have much to do with one continuous stream, but it did take sequences of the segments from multiple user's uploads and combine them for the different audio channels.
Most of the argument I think for what I'm seeing for voice AI is the code complexity, server complexity, interruptions, latency, drop outs, that kind of stuff, and I guess that's all good for companies with huge numbers of users and wanting to follow the work done at frontier labs. Also everyone wants the technology they sell to be the ones companies use.
Just weighing in that there could be cases where you record some audio, encode it, upload it, transcribe it, feed to an LLM, TTS the response and download it. You can stream it in segments or not to reduce latency. You can interrupt it with UI/DSP/NN and detect the end of a message similarly. Maybe for most smaller companies that would be enough? It feels like reading these different takes is that they create problems while solving problems, and maybe make the adoption of voice AI harder (and therefore slower) than it could be. I'm not any less naive than I was before though, but often its people like me tasked to implement this stuff.