When Fast Fourier Transform Meets Transformer for Image Restoration (2024)

See also: CosAE: Learnable Fourier Series for Image Restoration (2024)

https://sifeiliu.net/CosAE-page/

There have been some interesting advances in trying to add spectral information to the data that a learning architecture has at its disposal, but there are a couple roadblocks that I don’t think have been solved yet.

1. Complex-valued NNs are not an easy generalization of real ones.

2. A localization in one domain implies non-local behavior in the other (this is the Fourier uncertainty principle).

Fourier Neural Operators (FNOs) come close to what I want to see in this area but since they enforce sparsity in the spectral domain their application is necessarily limited.

[2024]