ECE 50024: Lecture

Professor Stanley H. Chan
Spring 2026

Week 1 Jan 19 - 21, 2021

Jan 19, 2021. Lecture 0. Course overview. Mathematics review.
- Live video recording (2021): Available in Brightspace.
- Please review Tutorials 0, 6, 7 (See below)

Jan 21, 2021. Lecture 1. Linear regression.
- Past videos (2020): (PDF) Linear Regression Concepts and Geometry (Video 1) (Video 2)
- See Chapter 7.1 of my book (URL)
- Please review Tutorials 1 and 2 (See below)
- Python demo for linear regression (URL)

Week 2, Jan 26 - 28, 2021

Jan 26, 2021. Lecture 2. Outliers in linear regression
- Hand writte note
- Live video recording (2021): Available in Brightspace.
- See Chapter 7.1 of my book (URL)
- Please review Tutorials 3 (See below)
- Python demo for linear regression with outliers (URL)

Jan 28, 2021. Lecture 3. Ridge and LASSO Regularization
- Hand writte note
- Past videos (2020): (PDF) Ridge and LASSO Regularization (Video 1) (Video 2)
- See Chapter 7.4 of my book (URL)
- Read the references in reading list page of the slides (PDF)
- Python demo for Ridge and LASSO (URL)

Week 3, Feb 2 - 4, 2021

Feb 2, 2021. Lecture 4. Optimization: Concepts
- Hand written note
- Past videos (2020): (PDF) Optimization 1: Optimality, Convexity, and Constraints (Video 1) (Video 2) (Video 3)
- Read the references in reading list page of the slides (PDF)
- Please review Tutorials 4 (See below)
- Python demo for CVX (URL)

Feb 4, 2021. Lecture 5. Optimization: Algorithms
- Hand written note
- Past videos (2020): (PDF) Optimization 2: Gradient Descent and Stochastic Gradient Descent (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)
- Please review Tutorials 4 (See below)
- Python demo for gradient descent (URL)

Week 4, Feb 9 - 11, 2021

Feb 9, 2021. Lecture 6. Linearly Separable
- Hand written note
- Past videos (2020): (PDF) Linear Separability (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)
- The proof of the separating hyperplane theorem is optional, but you are welcome to read.
- Python demo for constrained optimization (URL)

Feb 11, 2021. Lecture 7. Bayesian Classifier 1
- Hand written note
- Past videos (2020): (PDF) Linear Separability (Video 3)
- Past videos (2020): (PDF) Bayesian Decision Rule (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)
- Python demo for Bayesian decision rule (URL)

Week 5, Feb 16 - 18, 2021

Feb 16, 2021. Lecture 8. Bayesian Classifier 2
- Hand written note
- Past videos (2020): (PDF) Bayesian Decision Rule (Video 2) (Video 3)
- Past videos (2020): (PDF) Generative Method 2: Minimum Probility of Error Rule (Video) (First 30 minutes)
- Read the references in reading list page of the slides (PDF)
- Python demo for Bayesian decision rule (URL)

Feb 18, 2021. Lecture 9. Classification Error / ROC Curve
- Hand written note
- Past videos (2020): (PDF) Generative Method 2: Minimum Probility of Error Rule (Video) (Second 30 minutes)
- Read the references in reading list page of the slides (PDF)
- See Chapter Chapter 9.5 of my book (URL)
- Python demo for ROC (URL)

Week 6, Feb 23 - 25, 2021

Feb 23, 2021. Lecture 10. Estimating parameters
- Hand written note
- Past videos (2020): (PDF) Estimating Parameters (Video 1) (Video 2)
- Past videos (2020): (PDF) Bayesian Priors (Video 1)
- Read the references in reading list page of the slides (PDF) and (PDF)
- Read Chapter 8.1 and Chapter 8.3 of my book (URL)

Feb 25, 2021. Lecture 11. Logistic regression 1
- Hand written note
- Past videos (2020): (PDF) Loss and Convexity (Video 1) (Video 2) (Video 3)
- Read the references in reading list page of the slides (PDF) and (PDF)
- Python demo for logistic regression (URL)

Week 7, Mar 2 - Mar 4, 2021

Mar 2, 2021. Lecture 12. Logistic regression 2
- Hand written note
- Past videos (2020): (PDF) Algorithms and Interpretations (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF) and (PDF)
- Python demo for logistic regression (URL)

Mar 4, 2021. Lecture 13. Kernel trick
- Hand written note
- Past videos (2020): (PDF) Nonlinear transform, Kernel trick (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)
- Python demo for logistic regression (URL)

Week 8, Mar 9 - Mar 11, 2021

Mar 9, 2021. Lecture 14. Kernel trick + Probability Inequality
- Hand written note (kernel trick)
- Hand written note (probability inequality)
- Past videos (2020): (PDF) Probability Inequality (Video 1) (Video 2) (Video 3)
- Read the references in reading list page of the slides (PDF)

Mar 11, 2021. Lecture 15. Probability Inequality
- Hand written note (same as Lecture 14)
- Past videos (2020): (PDF) Probability Inequality (Video 1) (Video 2) (Video 3)
- Read the references in reading list page of the slides (PDF)
- Python demo for probability inequality (URL)

Week 9, Mar 16 - Mar 18, 2021

Mar 16, 2021. Lecture 16. Is learning feasible?
- Hand written note
- Past videos (2020): (PDF) Is Learning Feasible? (Video 1) (Video 2) (Video 3)
- Read the references in reading list page of the slides (PDF)

Mar 18, 2021. No class
- University reading day.

Week 10, Mar 23 - Mar 25, 2021

Mar 23, 2021. Lecture 17. Probably-Approximately Correct
- Hand written note
- Past videos (2020): (PDF) Probably Approximately Correct (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)

Mar 25, 2021. Lecture 18. Generalization bound
- Hand written note (same as Lecture 17)
- Past videos (2020): (PDF) Generalization (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)

Week 10, Mar 30 - Apr 1, 2021

Mar 30, 2021. Lecture 19. Growth function
- Hand written note
- Past videos (2020): (PDF) Growth Function (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)

Apr 1, 2021. Lecture 20. Growth function examples
- Hand written note
- Past videos (2020): (PDF) Growth Function (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)

Week 11, Apr 6 - Apr 8, 2021

Apr 6, 2021. Lecture 21. VC dimension
- Hand written note
- Additional note
- Past videos (2020): (PDF) VC Dimension (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)

Apr 8, 2021. Lecture 22. Bias and Variance
- Hand written note
- Past videos (2020): (PDF) Bias and Variance (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)

Week 12, Apr 13 - Apr 15, 2021

Apr 13, 2021. No class
- University reading day

Apr 15, 2021. Lecture 23. Overfitting
- Hand written note
- Past videos (2020): (PDF) Overfit (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)

Week 13, Apr 20 - Apr 22, 2021

Apr 20, 2021. Lecture 24. Into to neural network
- Hand written note
- Past videos (2020): (PDF) Feature Extraction 2: Hand-Crafted and Deep Features (Video 1) (Video 2) (Video 3)
- Read the references in reading list page of the slides (PDF)
- Python demo for MNIST dataset (URL)

Apr 22, 2021. Lecture 25. Convolutional structures and back propagation
- Hand written note
- Psat videos (2020): (PDF) Multi-Layer Perceptron and Back Propagation (Video 1) (Video 2)
- Read the references in reading list page of the slides (PDF)

Week 14, Apr 27 - Apr 29, 2021

Apr 27, 2021. Lecture 25. Generative adversarial networks

Apr 29, 2021. Lecture 26. Conclusion
- capsule network (PDF) (Video)
- (PDF) Conclusion: Practical Advices (Video 1) (Video 2)
- How to write a good paper
- What courses to take

Tutorials

Tutorial 0 Python Basics (PDF)
Tutorial 1 Linear Algebra Review (PDF)
Tutorial 2 Probability Review (PDF)
Tutorial 3 Linear Regression Examples (PDF)
Tutorial 4 Optimization Review (PDF)
Tutorial 5 Python for Image Processing (URL)
Tutorial 6 Python for Matrices (URL)
Tutorial 7 Python for Plotting (URL)