LIDIA: Lightweight learned image denoising with instance adaptation

Image denoising is a well studied problem with an extensive activity that has spread over several decades. Leading classical denoising methods are typically designed to exploit the inner structure in images by modeling local overlapping patches, and operating in an unsupervised fashion. In contrast, newcomers to this arena are supervised and universal neural-network-based methods that bypass this modeling altogether, targeting the inference goal directly and globally, tending to be deep and parameter heavy.This work proposes a novel lightweight learnable architecture for image denoising, using a combination of supervised and unsupervised training of it, the first aiming for a universal denoiser and the second for an instance adaptation. Our architecture embeds in it concepts taken from classical methods, leveraging patch processing, non-local self-similarity, representation sparsity and a multiscale treatment. Our proposed universal denoiser achieves near state-of-the-art results, while using a small fraction of the typical number of parameters. In addition, we introduce and demonstrate two highly effective ways for further boosting the denoising performance, by adapting this universal network to the input image. The code reproducing the results of this paper is available at https://github.com/grishavak/LIDIA-denoiser.