Internal versus external denoising—Benefits and bounds

Image denoising has been a popularly studied problem for several decades in image processing and low-level computer vision communities. Many effective denoising approaches, such as BM3D, utilize spatial redundancy of patches (relatively small, cropped windows) either within a single natural image, or within a large collection of natural images. In this chapter, we summarize our previous finding that “Internal-Denoising” (based on internal noisy patches) can outperform “External Denoising” (based on external clean patches), especially in the presence of high noise levels. We explain this phenomenon in terms of “Patch Signal-to-Noise Ratio” (PatchSNR), an inherent characteristic of a noisy patch that determines its preference of either internal or external denoising. We further experiment with the recent state-of-the-art convolutional residual neural network for Gaussian denoising. We show that it closes the gap on the previously reported external denoising bounds. We further compare its performance to internal local multi-scale Oracle (that has the same receptive field as the network). We show that for patches with low PatchSNR, the network does not manage to reconstruct the best “clean” patch that resides in the network’s receptive field. This suggests that the future challenge of denoising community is to train an image-specific CNN that will exploit local recurrence of patches, without relying on external examples, as was recently successfully done for super-resolution task. Combining such a model with external-based models may push PSNR bounds further up and improve denoising by ∼ 1–2 dB, especially for higher noise levels.