One Size Fits All: Can We Train One Denoiser for All Noise Levels?

International Conference on Machine Learning (ICML), 2020. (Acceptance rate = 21%)
Manuscript: https://arxiv.org/abs/2005.09627
Abhiram Gnansambandam and Stanley H. Chan

When training an estimator such as a neural network for tasks like image
denoising, it is generally preferred to train emph{one} estimator and apply
it to emph{all} noise levels. The de facto training protocol to achieve this
goal is to train the estimator with noisy samples whose noise levels are
uniformly distributed across the range of interest. However, why should we
allocate the samples uniformly? Can we have more training samples that are
less noisy, and fewer samples that are more noisy? What is the optimal
distribution? How do we obtain such a distribution? The goal of this paper is
to address this training sample distribution problem from a minimax risk
optimization perspective. We derive a dual ascent algorithm to determine the
optimal sampling distribution of which the convergence is guaranteed as long
as the set of admissible estimators is closed and convex. For estimators with
non-convex admissible sets such as deep neural networks, our dual formulation
converges to a solution of the convex relaxation. We discuss how the
algorithm can be implemented in practice. We evaluate the algorithm on linear
estimators and deep networks.

ConsensusNet: Optimal Combination of Image Denoisers

Given a set of image denoisers, each having a different denoising capability,
is there a provably optimal way of combining these denoisers to produce an
overall better result? An answer to this question is fundamental to designing
ensembles of weak estimators for complex scenes. In this paper, we present an
optimal procedure leveraging deep neural networks and convex optimization.
The proposed framework, called the Consensus Neural Network (CsNet),
introduces three new concepts in image denoising: (1) A deep neural network
to estimate the mean squared error (MSE) of denoised images without needing
the ground truths; (2) A provably optimal procedure to combine the denoised
outputs via convex optimization; (3) An image boosting procedure using a deep
neural network to improve contrast and to recover lost details of the
combined images. Experimental results show that CsNet can consistently
improve denoising performance for both deterministic and neural network
denoisers.

Quanta Image Sensor (QIS) is a single-photon image sensor that oversamples
the light field to generate binary measurements. Its single-photon
sensitivity makes it an ideal candidate for the next generation image sensor
after CMOS. However, image reconstruction of the sensor remains a challenging
issue. Existing image reconstruction algorithms are largely based on
optimization. In this paper, we present the first deep neural network
approach for QIS image reconstruction. Our deep neural network takes the
binary bit stream of QIS as input, learns the nonlinear transformation and
denoising simultaneously. Experimental results show that the proposed network
produces significantly better reconstruction results compared to existing
methods.