ECE 49595 - Cameras, Images, and Statistical Inverse Problems

Course Details

Lecture Hours: 3 Credits: 3

Counts as:

  • EE Elective
  • CMPE Selective - Special Content

Experimental Course Offered:

Spring 2022

Requisites:

ECE 20875, ECE 30200, MA 26500

Requisites by Topic:

Programming with Python, Probability, Linear Algebra

Catalog Description:

Image and signal processing of a digital camera; principles of image sensors; shot noise, read noise, dark current, and fixed pattern noise; statistical analysis of the noise; Gaussian and Poisson distributions; estimation techniques; maximum-likelihood estimation; maximum-a-posteriori estimation, minimum mean square estimation; formal definition of denoising; patch, reoccurrence and nonlocal techniques; kernel regression, symmetric smoothing filters, and graph denoisers; L2, L1, and total variation regularizations; fundamental limit of denoising; weak signals and the photon limit; variance stabilizing transform; motion estimation under noise; noise estimation.

Required Text(s):

  1. Intro to Probability for Data Science , Stanley Chan , Michigan Publishing , 2021 , ISBN No. 978-1-60785-746-4

Recommended Text(s):

  1. or download for free from Univ Michigan Press , ISBN No. 978-1-60785-747-1

Learning Outcomes:

A student who successfully fulfills the course requirements will have demonstrated:
  1. an ability to identify the source of Gaussian and Poisson noise, and describe their statistical properties. [1]
  2. an ability to apply maximum-likelihood, maximum-a-posteriori, and minimum mean square estimation methods for parameter estimation and signal recovery. [1]
  3. an ability to analyze the performance of signal recovery using statistical tools and experimental data. [1,6]
  4. an ability to modify algorithms for customized statistical inference problems. [7]

Lecture Outline:

Week Topics
1 From sensors to circuits to signals; Image and signal processing (ISP) pipeline of a digital camera in the 21st century, the technology behind CCD, CMOS, SPAD and QIS
2 Noise in sensors; dual nature of wave and photons, the physics of shot noise and read noise, Poisson-Gaussian affine model, quantile-quantile analysis of raw sensor data, dark current, fixed pattern noise, pixel non-uniformity
3 Gaussian statistics; Review of single-variate Gaussian, and extension to multi-variate Gaussian; mean, variance, skewness, kurtosis, sum of Gaussian, difference of Gaussian, linear transformation of Gaussian, Central limit theorem, Signal-to-noise ratio for additive noise models, High dimensional Gaussian PDF, covariance matrix and properties
4 Poisson statistics; Review of Poisson, and its variations, moment generating function, sum of Poisson, difference of Poisson, linear transformation of Poisson, limiting behavior of Poisson, law of small numbers, point process and inter-arrival time statistics, truncated Poisson and 1-bit sensors, Signal-to-noise ratio for Poisson, Signal-dependent noise model
5 Principles of maximum-likelihood; Basic concepts of maximum-likelihood, Likelihood function and its geometric interpretation, ML estimate for Gaussian and Poisson, properties of the ML estimators
6 Principles of maximum-a-posteriori; Basic concepts of MAP, The trio of likelihood, posterior and prior, Maximum-a-posterior estimation , properties of the MAP estimators, optimization methods for solving the MAP
7 Principles of minimum mean square estimation; Basic concepts of MMSE, David Donoho?'s formal definition of denoising, Minimum mean square estimation, Linear minimum mean square estimation, MMSE = Conditional expectation
8 Regularization for MAP-based denoising techniques; Regularization and optimization methods, ridge regularization, Sparsity and LASSO regularization, Total variation, ADMM algorithm
9 MAP based methods for Poisson noise reduction; Regularization and optimization methods, MAP + total variation for Poisson, closed form subproblem of the Poisson problem, the plug-and-play algorithms
10 Patch reoccurrence and non-local techniques; Images often have repeated patterns. How to find them, and how can they be used to denoise? Bilateral filter and recursive filter, Non-local means and block matching transform, Kernel regression, Symmetric smoothing filter, Doubly stochastic matrices and the Sinkhorn algorithm
11 Statistical methods for Poisson noise reduction; The principle of variance stabilizing transform for mean-dependent variances, Delta method and the variance stabilizing transform, Anscombe's 3/8 optimal Poisson transform, Optimal inversion and image denoising
12 How to estimate the noise level; Wavelet estimators, Stein unbiased risk estimators
13 Fundamental limit of noise removal; What is the fundamental limit of Gaussian noise removal? Can we achieve zero error? Is denoising dead? Patch complexity, finite pixel correlation, and optimal denoising
14 Handling moving scenes; Lucas-Kanade algorithm and its limit under noise, the controversy of tracking motion versus searching for patches, Burst and non-uniform sampler, Kernel prediction network, Advanced techniques by feature extraction
15 Learning-based methods; UNet, REDNet, FFDNet, Blade and RAISR, Ensemble methods and optimal combination, Can one denoiser work for all noise levels?

Engineering Design Content:

  • Analysis

Assessment Method:

Quizzes and Exams