alt Hacker NewsShow HN: Resonate – Low-latency, high-resolution spectral analysis
Last April I shared about my Resonate project here (https://news.ycombinator.com/item?id=43694157)
A lot has happened since: the work I presented in much more detail at last June's International Computer Music Conference (ICMC) got best paper award. I also gave a talk at the Audio Developer Conference in Bristol last November, the video is on YouTube).
This year's work, which I recently presented at this year's ICMC, starts with known techniques from the phase vocoder literature to build self-tuning filter banks that extract very efficiently the frequency components that are actually present in the input signal. Overview on the project website, more details in the papers, including applications to super-resolution spectrograms and re-synthesis experiments.
As many people have pointed out, none of the techniques I have used are new (some of them even have different names across different fields), but I haven't seen them applied together in this way, and to me the results are incredibly satisfying and sometimes look magical. See for example this demo: https://youtu.be/LasdoIJJkw8
Of course the best way to experience in person is through the free demo app: https://alexandrefrancois.org/Oscillators
Looking forward to feedback from the community!
Comments
A useful demo for this kind of tool would be a side-by-side on difficult inputs: fast pitch bends, dense chords, and low-SNR recordings. Latency is easy to appreciate visually, but robustness under messy audio is what usually decides whether spectral tools become part of a workflow. Even a small set of repeatable test clips would make the tradeoff much clearer.
Slowed down higher pitch example was nice to hear, as this is where often conventional methods are heavily artifacted
There are some piano tuners I've found who are a bit on the spectrum, who believe they can tune a piano in a way that no digital device can replicate. I'm skeptical, and would like to see how this method holds up against one of these savants.
And this is a tuner using same algorithm: https://apps.apple.com/us/app/resonance-chromatic-tuner/id16...
Also very old stuff :)
I have such app for more than 4 years. Your algorithm is not new - it is new for you only :) Here is the app: https://play.google.com/store/apps/details?id=com.bialamusic...
Nice. To the folks saying "nothing to see here" this appears to be a variation of filter-bank spectral analysis where each band varies in frequency to track "the" in-band sinusoid. Somewhat like a bank of PLLs each with its own tracking bandpass filter. By using IIR filters rather than FFTs you avoid the latency of buffering up a full frame of data before you can run the FFT analysis. I am curious how this handles input containing broadband transients. It might be interesting to use CIC filters rather than an IIR lowpass to get better time selectivity, but maybe that's already been addressed, I didn't read the papers.