Right. Of course there is some overhead for using WebGL vs native OpenGL calls, but I think that's only noticeable for AAA games. Web audio's extra latency does make it undesirable only for things that require quick response, like realtime synths and maybe DAWs.
Regarding libraries, I wonder if maybe there could be a cacheable package manager system for distributing wasm libraries to webapps.