ECE 50024: Lecture and Homework

Professor Stanley H. Chan
Spring 2025

13-Jan, 2025. Lecture 0. Course overview. Mathematics review.
- Lecture Slides (TBD)
- Homework
  - PDF (198KB)
  - Solution (257KB)
- Additional resources
  - Tutorial Python Basics (PDF)
  - Tutorial Python for Image Processing (URL)
  - Tutorial Python for Matrices (URL)
  - Tutorial Python for Plotting (URL)

15-Jan, 2025. Lecture 1. Linear Regression Concept and Geometry
- Lecture Slides PDF (574KB)
- Homework
  - PDF (229 KB)
  - Solution (289KB)
- Additional resources
  - Tutorial Linear Algebra Review (PDF)
  - See Chapter 7.1 of my book (URL)
  - Python demo for linear regression (URL)

17-Jan, 2025. Lecture 2. Underdetermined and Overdetermined Systems
- Lecture Slides PDF (846KB)
- Homework
  - PDF (380 KB)
  - Solution (396KB)
- Additional resources
  - See Chapter 7.1 of my book (URL)
  - Tutorial Linear Regression Examples (PDF)
  - Python demo for linear regression with outliers (URL)

20-Jan, 2025. No Class. MLK

22-Jan, 2025. Lecture 3. L1 and L2 Regularization
- Lecture Slides PDF (2.7MB)
- Homework
  - PDF (259 KB)
  - Solution: Login to Brightspace
- Additional resources
  - See Chapter 7.4 of my book (URL)
  - Read the references in reading list page of the slides.
  - Python demo for Ridge and LASSO (URL)

24-Jan, 2025. Lecture 4. Kernel Methods
- Lecture Slides PDF (1.1MB)
- Homework PDF (200 KB)
  - Solution: Login to Brightspace

27-Jan, 2025. Lecture 5. Optimization basics; optimality; convexity
- Lecture Slides PDF (1.8MB)
- Homework
  - PDF (204 KB)
  - Solution: Login to Brightspace
- Additional resources
  - Tutorial 4 Optimization Review (PDF)
  - Read the references in reading list page of the slides.
  - Python demo for CVX (URL)

29-Jan, 2025. Lecture 6. Gadient descent
- Lecture Slides PDF (939KB)
- Homework
  - PDF (208 KB)
  - Solution: Login to Brightspace
- Additional resources
  - Read the references in reading list page of the slides.
  - Tutorial 4 Optimization Review (PDF)
  - Python demo for gradient descent (URL)

31-Jan, 2025. Lecture 7. Linear Discriminant Analysis
- Lecture Slides PDF (663KB)
- Homework
  - PDF (213 KB)
  - Solution: Login to Brightspace
- Additional resources
  - Read the references in reading list page of the slides
  - Python demo for linear separability (URL)

3-Feb, 2025. Lecture 8. Linearly Separable
- Lecture Slides PDF (750KB)
- Homework
  - PDF (120 KB)
  - Solution: Login to Brightspace
- Additional resources
  - Read the references in reading list page of the slides
  - Python demo for linear separability (URL)

5-Feb, 2025. Lecture-A. Guest Lecture: Feature extraction and convolution

7-Feb, 2025. Monthly Test 1

10-Feb, 2025. Lecture 9 Bayesian Decision
- Lecture Slides PDF (896KB)
- Homework
  - PDF (234 KB)
  - Solution: Login to Brightspace
- Additional resources
  - Tutorial 2 Probability Review (PDF)

12-Feb, 2025. Lecture 10 Probability of Error
- Lecture Slides PDF (710KB)
- Homework
  - PDF (247 KB)
  - Solution: Login to Brightspace

14-Feb, 2025. Lecture 11 Model parameter estimation
- Lecture Slides PDF (779KB)
- Homework
  - PDF (215 KB)
  - Solution: Login to Brightspace

17-Feb, 2025. Lecture 12 Connecting Bayesian with regression
- Lecture Slides PDF (676KB)
- Homework
  - TBD
  - Solution: Login to Brightspace

19-Feb, 2025. Lecture 13 Why are classifier vulnerable to attacks?
- Lecture Slides PDF (5.7MB)
- Homework
  - PDF (167 KB)
  - Solution: Login to Brightspace

21-Feb, 2025. Lecture 14 Min-norm perturbuation
- Lecture Slides PDF (487KB)
- Homework
  - PDF (184 KB)
  - Solution: Login to Brightspace

24-Feb, 2025. Lecture 15 Max-loss and regularized perturbation
- Lecture Slides PDF (1.5MB)
- Homework
  - PDF (196 KB)
  - Solution: Login to Brightspace

26-Feb, 2025. Lecture 16 Improving robustness
- Lecture Slides PDF (2.3MB)
- Homework
  - PDF (334 KB)
  - Solution: Login to Brightspace

28-Feb, 2025. Lecture 17 Variational autoencoder
- Lecture Slides PDF (1.8MB)
- Homework
  - PDF
  - Solution: Login to Brightspace

3-Mar, 2025. Lecture 18 Evidence lower bound
- Lecture Slides PDF (485KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

5-Mar, 2025. Lecture 19 Parameterization Trick
- Lecture Slides PDF (859KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

7-Mar, 2025. Monthly Test 2

10-Mar, 2025. Lecture 20 DDPM building blocks
- Lecture Slides PDF (2.3MB)
- Homework
  - PDF
  - Solution: Login to Brightspace

12-Mar, 2025. Lecture 21 DDPM evidence lower bound
- Lecture Slides PDF (560KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

14-Mar, 2025. Lecture 22 DDPM: Denoising?
- Lecture Slides PDF (591KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

17-Mar, 2025. Spring Break

19-Mar, 2025. Spring Break

21-Mar, 2025. Spring Break

24-Mar, 2025. Lecture 23 Training and Inference of DDPM
- Lecture Slides PDF (1.7MB)
- Homework
  - PDF
  - Solution: Login to Brightspace

26-Mar, 2025. Lecture 24 Score matching and Langevin Sampling
- Lecture Slides PDF (1.7MB)
- Homework
  - PDF
  - Solution: Login to Brightspace

28-Mar, 2025. Lecture 25 Score Matching Techniques
- Lecture Slides PDF (573KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

31-Mar, 2025. Lecture 26 Stochastic Differential Equation
- Lecture Slides PDF (1.8MB)
- Homework
  - PDF
  - Solution: Login to Brightspace

2-Apr, 2025. Lecture 27 Brownian Motion
- Lecture Slides PDF (527KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

4-Apr, 2025. Monthly Test 3

7-Apr, 2025. Lecture 28 Fokker-Planck Equation
- Lecture Slides PDF (394KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

9-Apr, 2025. Lecture 29 Probability Inequality
- Lecture Slides PDF (438KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

11-Apr, 2025. Lecture 30 Is learning always feasible?
- Lecture Slides PDF (753KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

14-Apr, 2025. Lecture 31 Generalization bound
- Lecture Slides PDF (394KB)
- Homework TBD
  - Solution: Login to Brightspace

16-Apr, 2025. Lecture 32 Growth function
- Lecture Slides PDF (472KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

18-Apr, 2025. Lecture 33 VC dimension
- Lecture Slides PDF (520KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

21-Apr, 2025. Lecture 34 Sample and Model Complexity
- Lecture Slides PDF (441KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

23-Apr, 2025. Lecture 35 Bias and variance
- Lecture Slides PDF (621KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

25-Apr, 2025. Lecture 36 Overfitting
- Lecture Slides PDF (770KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

28-Apr, 2025. Lecture 37 Regularization
- Lecture Slides PDF (546KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

30-Apr, 2025. Lecture 38 Validation
- Lecture Slides PDF (685KB)
- Homework
  - PDF
  - Solution: Login to Brightspace

2-May, 2025. Monthly Test 4 (Or during final exam week. TBD)

Tutorials

Tutorial 0 Python Basics (PDF)
Tutorial 1 Linear Algebra Review (PDF)
Tutorial 2 Probability Review (PDF)
Tutorial 3 Linear Regression Examples (PDF)
Tutorial 4 Optimization Review (PDF)
Tutorial 5 Python for Image Processing (URL)
Tutorial 6 Python for Matrices (URL)
Tutorial 7 Python for Plotting (URL)