Funny, I guess this idea is in the air (har har).

I've made a couple things along these lines over the past year. One is a fun, Halloween-themed toy:

https://blog.cochlea.xyz/theremin.html

The other is a (slightly) more serious exploration of multivariate time signals as a control signal for an instrument. In the latter case, the instrument is learned/inferred from a segment of input audio:

https://blog.cochlea.xyz/resonancemodel.html

This is a really neat idea! Like others here, I'm not a huge fan of the hyper-dense and dripping-with-ai UI, though.