ECE 50024: Lecture

Week 1 Jan 19 - 21, 2021

• Jan 19, 2021. Lecture 0. Course overview. Mathematics review.

  • Live video recording (2021): Available in Brightspace.

  • Please review Tutorials 0, 6, 7 (See below)
    

• Jan 21, 2021. Lecture 1. Linear regression.

  • Past videos (2020): (PDF) Linear Regression Concepts and Geometry (Video 1) (Video 2)

  • See Chapter 7.1 of my book (URL)

  • Please review Tutorials 1 and 2 (See below)

  • Python demo for linear regression (URL)
    
    

Week 2, Jan 26 - 28, 2021

• Jan 26, 2021. Lecture 2. Outliers in linear regression

  • Hand writte note

  • Live video recording (2021): Available in Brightspace.

  • See Chapter 7.1 of my book (URL)

  • Please review Tutorials 3 (See below)

  • Python demo for linear regression with outliers (URL)
    

• Jan 28, 2021. Lecture 3. Ridge and LASSO Regularization

  • Hand writte note

  • Past videos (2020): (PDF) Ridge and LASSO Regularization (Video 1) (Video 2)

  • See Chapter 7.4 of my book (URL)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for Ridge and LASSO (URL)
    
    

Week 3, Feb 2 - 4, 2021

• Feb 2, 2021. Lecture 4. Optimization: Concepts

  • Hand written note

  • Past videos (2020): (PDF) Optimization 1: Optimality, Convexity, and Constraints (Video 1) (Video 2) (Video 3)

  • Read the references in reading list page of the slides (PDF)

  • Please review Tutorials 4 (See below)

  • Python demo for CVX (URL)
    

• Feb 4, 2021. Lecture 5. Optimization: Algorithms

  • Hand written note

  • Past videos (2020): (PDF) Optimization 2: Gradient Descent and Stochastic Gradient Descent (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)

  • Please review Tutorials 4 (See below)

  • Python demo for gradient descent (URL)
    
    

Week 4, Feb 9 - 11, 2021

• Feb 9, 2021. Lecture 6. Linearly Separable

  • Hand written note

  • Past videos (2020): (PDF) Linear Separability (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)

  • The proof of the separating hyperplane theorem is optional, but you are welcome to read.

  • Python demo for constrained optimization (URL)
    

• Feb 11, 2021. Lecture 7. Bayesian Classifier 1

  • Hand written note

  • Past videos (2020): (PDF) Linear Separability (Video 3)

  • Past videos (2020): (PDF) Bayesian Decision Rule (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for Bayesian decision rule (URL)
    
    

Week 5, Feb 16 - 18, 2021

• Feb 16, 2021. Lecture 8. Bayesian Classifier 2

  • Hand written note

  • Past videos (2020): (PDF) Bayesian Decision Rule (Video 2) (Video 3)

  • Past videos (2020): (PDF) Generative Method 2: Minimum Probility of Error Rule (Video) (First 30 minutes)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for Bayesian decision rule (URL)
    

• Feb 18, 2021. Lecture 9. Classification Error / ROC Curve

  • Hand written note

  • Past videos (2020): (PDF) Generative Method 2: Minimum Probility of Error Rule (Video) (Second 30 minutes)

  • Read the references in reading list page of the slides (PDF)

  • See Chapter Chapter 9.5 of my book (URL)

  • Python demo for ROC (URL)
    
    

Week 6, Feb 23 - 25, 2021

• Feb 23, 2021. Lecture 10. Estimating parameters

  • Hand written note

  • Past videos (2020): (PDF) Estimating Parameters (Video 1) (Video 2)

  • Past videos (2020): (PDF) Bayesian Priors (Video 1)

  • Read the references in reading list page of the slides (PDF) and (PDF)

  • Read Chapter 8.1 and Chapter 8.3 of my book (URL)
    

• Feb 25, 2021. Lecture 11. Logistic regression 1

  • Hand written note

  • Past videos (2020): (PDF) Loss and Convexity (Video 1) (Video 2) (Video 3)

  • Read the references in reading list page of the slides (PDF) and (PDF)

  • Python demo for logistic regression (URL)
    
    

Week 7, Mar 2 - Mar 4, 2021

• Mar 2, 2021. Lecture 12. Logistic regression 2

  • Hand written note

  • Past videos (2020): (PDF) Algorithms and Interpretations (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF) and (PDF)

  • Python demo for logistic regression (URL)
    

• Mar 4, 2021. Lecture 13. Kernel trick

  • Hand written note

  • Past videos (2020): (PDF) Nonlinear transform, Kernel trick (Video 1) (Video 2)
    

  • Read the references in reading list page of the slides (PDF)

  • Python demo for logistic regression (URL)
    
    

Week 8, Mar 9 - Mar 11, 2021

• Mar 9, 2021. Lecture 14. Kernel trick + Probability Inequality

  • Hand written note (kernel trick)

  • Hand written note (probability inequality)

  • Past videos (2020): (PDF) Probability Inequality (Video 1) (Video 2) (Video 3)
    

  • Read the references in reading list page of the slides (PDF)
    

• Mar 11, 2021. Lecture 15. Probability Inequality

  • Hand written note (same as Lecture 14)

  • Past videos (2020): (PDF) Probability Inequality (Video 1) (Video 2) (Video 3)
    

  • Read the references in reading list page of the slides (PDF)

  • Python demo for probability inequality (URL)
    
    

Week 9, Mar 16 - Mar 18, 2021

• Mar 16, 2021. Lecture 16. Is learning feasible?

  • Hand written note

  • Past videos (2020): (PDF) Is Learning Feasible? (Video 1) (Video 2) (Video 3)
    

  • Read the references in reading list page of the slides (PDF)
    

• Mar 18, 2021. No class

  • University reading day.
    
    

Week 10, Mar 23 - Mar 25, 2021

• Mar 23, 2021. Lecture 17. Probably-Approximately Correct

  • Hand written note

  • Past videos (2020): (PDF) Probably Approximately Correct (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    

• Mar 25, 2021. Lecture 18. Generalization bound

  • Hand written note (same as Lecture 17)

  • Past videos (2020): (PDF) Generalization (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    
    

Week 10, Mar 30 - Apr 1, 2021

• Mar 30, 2021. Lecture 19. Growth function

  • Hand written note

  • Past videos (2020): (PDF) Growth Function (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    

• Apr 1, 2021. Lecture 20. Growth function examples

  • Hand written note

  • Past videos (2020): (PDF) Growth Function (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    
    

Week 11, Apr 6 - Apr 8, 2021

• Apr 6, 2021. Lecture 21. VC dimension

  • Hand written note

  • Additional note

  • Past videos (2020): (PDF) VC Dimension (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    

• Apr 8, 2021. Lecture 22. Bias and Variance

  • Hand written note

  • Past videos (2020): (PDF) Bias and Variance (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    
    

Week 12, Apr 13 - Apr 15, 2021

• Apr 13, 2021. No class

  • University reading day

• Apr 15, 2021. Lecture 23. Overfitting

  • Hand written note

  • Past videos (2020): (PDF) Overfit (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    
    

Week 13, Apr 20 - Apr 22, 2021

• Apr 20, 2021. Lecture 24. Into to neural network

  • Hand written note

  • Past videos (2020): (PDF) Feature Extraction 2: Hand-Crafted and Deep Features (Video 1) (Video 2) (Video 3)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for MNIST dataset (URL)
    

• Apr 22, 2021. Lecture 25. Convolutional structures and back propagation

  • Hand written note

  • Psat videos (2020): (PDF) Multi-Layer Perceptron and Back Propagation (Video 1) (Video 2)
    

  • Read the references in reading list page of the slides (PDF)
    
    

Week 14, Apr 27 - Apr 29, 2021

• Apr 27, 2021. Lecture 25. Generative adversarial networks

  • The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

  • Tutorial about GAN

  • Demo for GAN

  • Detecting Photoshopped Faces by Scripting Photoshop

• Apr 29, 2021. Lecture 26. Conclusion

  • capsule network (PDF) (Video)

  • (PDF) Conclusion: Practical Advices (Video 1) (Video 2)

  • How to write a good paper

  • What courses to take
    
    

Tutorials

• Tutorial 0 Python Basics (PDF)

• Tutorial 1 Linear Algebra Review (PDF)

• Tutorial 2 Probability Review (PDF)

• Tutorial 3 Linear Regression Examples (PDF)

• Tutorial 4 Optimization Review (PDF)

• Tutorial 5 Python for Image Processing (URL)

• Tutorial 6 Python for Matrices (URL)

• Tutorial 7 Python for Plotting (URL)