Deep Learning

ConsensusNet: Optimal Combination of Image Denoisers


Given a set of image denoisers, each having a different denoising capability, is there a provably optimal way of combining these denoisers to produce an overall better result? An answer to this question is fundamental to designing ensembles of weak estimators for complex scenes. In this paper, we present an optimal procedure leveraging deep neural networks and convex optimization. The proposed framework, called the Consensus Neural Network (CsNet), introduces three new concepts in image denoising: (1) A deep neural network to estimate the mean squared error (MSE) of denoised images without needing the ground truths; (2) A provably optimal procedure to combine the denoised outputs via convex optimization; (3) An image boosting procedure using a deep neural network to improve contrast and to recover lost details of the combined images. Experimental results show that CsNet can consistently improve denoising performance for both deterministic and neural network denoisers.


  1. Joon Hee Choi, Omar A. Elgendy and Stanley H. Chan, ‘‘Optimal Combination of Image Denoisers’’, submitted. (Supplementary Material)

Deep Learning for QIS Image Reconstruction


Quanta Image Sensor (QIS) is a single-photon image sensor that oversamples the light field to generate binary measurements. Its single-photon sensitivity makes it an ideal candidate for the next generation image sensor after CMOS. However, image reconstruction of the sensor remains a challenging issue. Existing image reconstruction algorithms are largely based on optimization. In this paper, we present the first deep neural network approach for QIS image reconstruction. Our deep neural network takes the binary bit stream of QIS as input, learns the nonlinear transformation and denoising simultaneously. Experimental results show that the proposed network produces significantly better reconstruction results compared to existing methods.


  1. Joon Hee Choi, Omar A. Elgendy and Stanley H. Chan, ‘‘Image reconstruction for Quanta Image Sensors using deep neural networks’’, IEEE ICASSP, pp. 6543–6547, Calgary, Canada, Apr. 2018.