# ECE 495: Cameras, Images, and Statistical Inverse Problems

Professor Stanley H. Chan, Purdue University, Spring 2022

## Announcement

• 01/06/2022 Classroom changed to MSEE B010.

• 10/01/2021 Course website launched.

## Course Information

Lecture: MWF 10:30am - 11:20am
Room: MSEE B010 (in-person)
Lectures will be recorded via BoilerCast, and will be made available in Brightspace (subject to delays).

Instructor: Professor Stanley H. Chan
Room: MSEE 338
Email: stanleychan AT purdue DOT edu
Office Hour: After class, and by appointment

Teaching Assistants:
Nick Chimitt, nchimitt AT purdue DOT edu
Guanzhe Hong, hong288 AT purdue DOT edu
Office Hour: By appointment

## Note

### Part 1: Understanding Your Camera

• Lecture Note 1-1: Cameras in the 21st century (PDF, 6MB)

• Lecture Note 1-2: Digital image sensors (PDF, 6MB)

• Lecture Note 1-3: Noise

• Lecture Note 1-4: Signal-to-Noise Ratio (PDF, 580KB)

• Lecture Note 1-5: Dynamic Range (PDF, 2.3MB)

### Part 2: Probability and Statistics

• Lecture Note 2-1: Gaussian Random Variables (PDF, 500KB)

• Lecture Note 2-2: Central Limit Theorem (PDF, 820KB)

• Lecture Note 2-3: High-dimensional Random Variables (PDF, 830KB)

• Lecture Note 2-4: Basics of Poisson random variables (PDF, 500KB)

• Lecture Note 2-5: Physics of Photon Arrivals (PDF, 540KB)

• Lecture Note 2-6: Single-Photon and Low Bit-Depth Statistics (PDF, 900KB)

### Part 3: Estimation Techniques

• Lecture Note 3-1: Maximum-Likelihood Estimation (PDF, 1.6MB)

• Lecture Note 3-2: Properties of ML Estimation (PDF, 480KB)

• Lecture Note 3-3: Maximum-A-Posteriori Estimation (PDF, 588KB)

• Lecture Note 3-4: Minimum Mean-Square Estimation (PDF, 432KB)

### Part 4: Denoising

• Lecture Note 4-1: Linear Inverse Problems (PDF, 1MB)

• Lecture Note 4-2: ADMM Algorithm (PDF, 800KB)

• Lecture Note 4-3: Patch Reoccurence

• Lecture Note 4-4: Smoothing Filters

• Lecture Note 4-5: Variance Stabilizing Transforms

• Lecture Note 4-6: Is Denoising Dead?

### Part 5: Learning-based Methods

(Not sure if we will have time to get here.)

• Lecture Note 5-1: Network Unrolling

• Lecture Note 5-2: Knowledge Distillation

• Lecture Note 5-3: One-size-fit-all

• Lecture Note 5-4: Dynamic Scenes

## Homework (30%)

There will be six homework. I will drop the worst one. If there is a programming problem, you can choose whatever language you like: MATLAB, Python, Julia, C (I wouldn't recommend it though).

Late homework will not be accepted.

• Homework 0 (This homework does not carry points.)

• Due Jan 12, 2022, 11:59pm Eastern Time

• Homework 1

• Due Jan 26, 2022, Wednesday, 11:59pm Eastern Time

• Homework 2

• Due Feb 11, 2022, Friday, 11:59pm Eastern Time

• Homework 3

• Due March 2, 2022, Wednesday, 11:59pm Eastern Time

• Homework 4

• Due March 23, 2022, Wednesday, 11:59pm Eastern Time

• Homework 5

• Due April 8, 2022, Friday, 11:59pm Eastern Time

• Homework 6

• Due April 27, 2022, Wednesday, 11:59pm Eastern Time

## Quiz (30%)

There will be six quizes. I will drop the worst one. Each quiz is 30 minutes long. The quizes are conducted right after the due date of the homework. You will be given a 72 hour window to complete the 30-min quiz, completely online. During the quiz, I will ask you lecture questions. I will also ask you homework questions. For example, if in the homework I ask you to plot a figure, I may ask you to change a parameter and re-plot the figure. Quizes will be open-book, open-note, open-computer. However, with only 30 minutes, you probably will not have time to read anything besides answering the questions. So, please do the homework.

• Quiz 0 (This quiz does not carry any point)

• Start: Jan 1, 2022 12:01am Eastern Time

• End: Jan 12, 2022 11:59pm Eastern Time

• Quiz 1

• Start: Jan 27, 2022 12:01am Eastern Time (Thursday)

• End: Jan 30, 2022 11:59pm Eastern Time (Saturday)

• Quiz 2

• Start: Feb 12, 2022 12:01am Eastern Time (Saturday)

• End: Feb 14, 2022 11:59pm Eastern Time (Monday)

• Quiz 3

• Start: March 3, 2022 12:01am Eastern Time (Thursday)

• End: March 5, 2022 11:59pm Eastern Time (Saturday)

• Quiz 4

• Start: March 24, 2022 12:01am Eastern Time (Thursday)

• End: March 26, 2022 11:59pm Eastern Time (Saturday)

• Quiz 5

• Start: April 9, 2022 12:01am Eastern Time (Saturday)

• End: April 11, 2022 11:59pm Eastern Time (Monday)

• Quiz 6

• Start: April 28, 2022 12:01am Eastern Time (Thursday)

• End: April 30, 2022 11:59pm Eastern Time (Saturday)

## Seminar Reports (30%)

Attend any THREE of the following seminars. Write a report of no more than 2 pages, 11 points, Times Roman, 1 inch margin. The grading criteria is based on how much thinking you put into each report. Show me that you have thought about these questions.

• Jan 17: Eric Fossum, Quanta Image Sensors.

• Date: Jan 17, 2022. 10am Eastern Time.

• Q1: Please summarize the talk in 5-7 sentences.

• Q2: What is the potential benefit and weakness of QIS relative to SPAD?

• Q3: Why is photon resolving an important problem for imaging?

• Q4: According to the speaker, what are the societal issues camera engineers should be aware of?

• Q5: What is the role of signal processing in QIS?

• Q6: What made you exited about / inspired you when you attend the talk?

• Feel free to insert pictures, graphs, etc to illustrate your points.

• Report is due on Jan 31, 2022 11:59pm (submit to gradescope)

• Jan 26: Bill Freeman

• Date: Jan 26, 2022. 2:30pm Eastern Time.

• Q1. Please summarize the talk in 5-7 sentences.

• Q2. What motivates Dr. Freeman to consider this problem?

• Q3. Can you describe the basic principle of capturing the image of the earth from the earth?

• Q4. What are the limitations of such an approach, ie, how much resolution can we ultimately get?

• Q5. How is the moon camera related to seeing around the corner?

• Q6. What made you exited about / inspired you when you attend the talk?

• Feel free to insert pictures, graphs, etc to illustrate your points.

• Report is due on Feb 11, 2022 11:59pm (submit to gradescope)

• Feb 15: Sanjeev Koppal

• Date: Feb 14, 2022. 11:30pm Eastern Time.

• Zoom: Check brightspace

• Q1. Summarize the talk in 5-7 sentences

• Q2. Based on your understanding of the talk (and other online sources), what is bio-inspired vision and how far are we from the goal?

• Q3. For the LiDAR project, what is the principle the speaker used to adaptively sense the depth? There are two parts of this question: The first one is about the mirror and the other one is about the control mechanism. Describe both.

• Q4. For the adaptive camera project, the speaker's idea is to allocate bits to more important objects when the transmission budget is limited. Speaking of this technology, what are the pros and cons you can think of? Why?

• Q5. What is the most inspiring part of the talk, and what did you learn?

• Report is due on March 1, 2022 11:59pm (submit to gradescope)

• Mar 1: Yaniv Romano

• Date: Mar 1, 2022. 10am Eastern Time

• Zoom: Check brightspace

• Q1: Summarize the talk in 5-7 sentences

• Q2: According to Professor Romano and based on your readings online, why is important to have an accurate of the confidence interval?

• Q3: He mentioned that today's models are mostly black boxes, and discussed the concept of a wrapper function. What is it, and why is it relevant?

• Q4: In your own words, describe the quantile estimation procedure he presented. What statistical guarantees can his procedure offer?

• Q5: He showed an example of the 2020 presidential election. Can you describe how his technique is used?

• Q6. What is the most inspiring part of the talk, and what did you learn?

• Report is due on March 23, 2022 11:59pm (submit to gradescope)

• Mar 22: Joyce Farrell

• Date: Mar 22, 2022. 1pm Eastern Time

• Zoom: Check brightspace

• Q1: Summarize the talk in 5-7 sentences

• Q2: Why is physics-based simulation an important tool for the camera industry?

• Q3: What are the main tools Dr Farrell used to do the simulation? Can you name a few steps? It might be good to briefly read her paper and summarize the key ideas.

• Q4: How do they measure the correctness of the simulator? She mentioned a few key metrics. Can you summarize a few here?

• Q5: What are the applications of the simulator, and how is the simulator used in those scenarios? What does the simulator offer?

• Q6. What is the most inspiring part of the talk, and what did you learn?

• Apr 13: Kyros Kutulakos

• Date: Apr 13, 2022. 12pm Eastern Time

• Zoom: Check brightspace

• Q1: Summarize the talk in 5-7 sentences

• Q2: Can describe the basic principle of light transport, and how did the presenter measure lights from different optical paths?

• Q3: What are some of the limitations of the technology?

• Q4: What can you do with the technology? Mention a few possible applications talked by the speaker.

• Q5: He mentioned the concept of dual pixel. What can you use it for?

• Q6. What is the most inspiring part of the talk, and what did you learn?

These talks are part of Purdue computational imaging seminar.

## Class Participation (10%)

This is a small class and so I will be able to remember everyone of you, and you will have the chance to interact a lot with me. The purpose of the class participation is to encourage intellectual discussions. Come and discuss things with me. There are a few ways I will use to check your participation:

• Come to class (I will not take attendence, but with such a small class I will remember you.)

• Talk to the TAs

You do not need to do everything listed above, nor I will have a systematic way of checking you. Basically, as long as you do not hide yourself, you will be fine for this portion of the grade.

## COVID and Health Related Issues

So here is the rule of thumb:

• Follow Protect Purdue instructions. If you don't know what I am talking about, visit https://protect.purdue.edu/. Follow everything they say.