ECE 50024: Lecture

The schedule below was for Spring 2021. For students who are enrolled in the Fall 2023, you should be able to see the new schedule in Brightspace. Please follow that. The one shown here are for external visitors.

Week 1 Jan 19 - 21, 2021

• Jan 19, 2021. Lecture 0. Course overview. Mathematics review.

  • Live video recording (2021): Available in Brightspace.

  • Please review Tutorials 0, 6, 7 (See below)
    

• Jan 21, 2021. Lecture 1. Linear regression.

  • Past videos (2020): (PDF) Linear Regression Concepts and Geometry (Video 1) (Video 2)

  • See Chapter 7.1 of my book (URL)

  • Please review Tutorials 1 and 2 (See below)

  • Python demo for linear regression (URL)
    
    

Week 2, Jan 26 - 28, 2021

• Jan 26, 2021. Lecture 2. Outliers in linear regression

  • Hand writte note

  • Live video recording (2021): Available in Brightspace.

  • See Chapter 7.1 of my book (URL)

  • Please review Tutorials 3 (See below)

  • Python demo for linear regression with outliers (URL)
    

• Jan 28, 2021. Lecture 3. Ridge and LASSO Regularization

  • Hand writte note

  • Past videos (2020): (PDF) Ridge and LASSO Regularization (Video 1) (Video 2)

  • See Chapter 7.4 of my book (URL)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for Ridge and LASSO (URL)
    
    

Week 3, Feb 2 - 4, 2021

• Feb 2, 2021. Lecture 4. Optimization: Concepts

  • Hand written note

  • Past videos (2020): (PDF) Optimization 1: Optimality, Convexity, and Constraints (Video 1) (Video 2) (Video 3)

  • Read the references in reading list page of the slides (PDF)

  • Please review Tutorials 4 (See below)

  • Python demo for CVX (URL)
    

• Feb 4, 2021. Lecture 5. Optimization: Algorithms

  • Hand written note

  • Past videos (2020): (PDF) Optimization 2: Gradient Descent and Stochastic Gradient Descent (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)

  • Please review Tutorials 4 (See below)

  • Python demo for gradient descent (URL)
    
    

Week 4, Feb 9 - 11, 2021

• Feb 9, 2021. Lecture 6. Linearly Separable

  • Hand written note

  • Past videos (2020): (PDF) Linear Separability (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)

  • The proof of the separating hyperplane theorem is optional, but you are welcome to read.

  • Python demo for constrained optimization (URL)
    

• Feb 11, 2021. Lecture 7. Bayesian Classifier 1

  • Hand written note

  • Past videos (2020): (PDF) Linear Separability (Video 3)

  • Past videos (2020): (PDF) Bayesian Decision Rule (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for Bayesian decision rule (URL)
    
    

Week 5, Feb 16 - 18, 2021

• Feb 16, 2021. Lecture 8. Bayesian Classifier 2

  • Hand written note

  • Past videos (2020): (PDF) Bayesian Decision Rule (Video 2) (Video 3)

  • Past videos (2020): (PDF) Generative Method 2: Minimum Probility of Error Rule (Video) (First 30 minutes)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for Bayesian decision rule (URL)
    

• Feb 18, 2021. Lecture 9. Classification Error / ROC Curve

  • Hand written note

  • Past videos (2020): (PDF) Generative Method 2: Minimum Probility of Error Rule (Video) (Second 30 minutes)

  • Read the references in reading list page of the slides (PDF)

  • See Chapter Chapter 9.5 of my book (URL)

  • Python demo for ROC (URL)
    
    

Week 6, Feb 23 - 25, 2021

• Feb 23, 2021. Lecture 10. Estimating parameters

  • Hand written note

  • Past videos (2020): (PDF) Estimating Parameters (Video 1) (Video 2)

  • Past videos (2020): (PDF) Bayesian Priors (Video 1)

  • Read the references in reading list page of the slides (PDF) and (PDF)

  • Read Chapter 8.1 and Chapter 8.3 of my book (URL)
    

• Feb 25, 2021. Lecture 11. Logistic regression 1

  • Hand written note

  • Past videos (2020): (PDF) Loss and Convexity (Video 1) (Video 2) (Video 3)

  • Read the references in reading list page of the slides (PDF) and (PDF)

  • Python demo for logistic regression (URL)
    
    

Week 7, Mar 2 - Mar 4, 2021

• Mar 2, 2021. Lecture 12. Logistic regression 2

  • Hand written note

  • Past videos (2020): (PDF) Algorithms and Interpretations (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF) and (PDF)

  • Python demo for logistic regression (URL)
    

• Mar 4, 2021. Lecture 13. Kernel trick

  • Hand written note

  • Past videos (2020): (PDF) Nonlinear transform, Kernel trick (Video 1) (Video 2)
    

  • Read the references in reading list page of the slides (PDF)

  • Python demo for logistic regression (URL)
    
    

Week 8, Mar 9 - Mar 11, 2021

• Mar 9, 2021. Lecture 14. Kernel trick + Probability Inequality

  • Hand written note (kernel trick)

  • Hand written note (probability inequality)

  • Past videos (2020): (PDF) Probability Inequality (Video 1) (Video 2) (Video 3)
    

  • Read the references in reading list page of the slides (PDF)
    

• Mar 11, 2021. Lecture 15. Probability Inequality

  • Hand written note (same as Lecture 14)

  • Past videos (2020): (PDF) Probability Inequality (Video 1) (Video 2) (Video 3)
    

  • Read the references in reading list page of the slides (PDF)

  • Python demo for probability inequality (URL)
    
    

Week 9, Mar 16 - Mar 18, 2021

• Mar 16, 2021. Lecture 16. Is learning feasible?

  • Hand written note

  • Past videos (2020): (PDF) Is Learning Feasible? (Video 1) (Video 2) (Video 3)
    

  • Read the references in reading list page of the slides (PDF)
    

• Mar 18, 2021. No class

  • University reading day.
    
    

Week 10, Mar 23 - Mar 25, 2021

• Mar 23, 2021. Lecture 17. Probably-Approximately Correct

  • Hand written note

  • Past videos (2020): (PDF) Probably Approximately Correct (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    

• Mar 25, 2021. Lecture 18. Generalization bound

  • Hand written note (same as Lecture 17)

  • Past videos (2020): (PDF) Generalization (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    
    

Week 10, Mar 30 - Apr 1, 2021

• Mar 30, 2021. Lecture 19. Growth function

  • Hand written note

  • Past videos (2020): (PDF) Growth Function (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    

• Apr 1, 2021. Lecture 20. Growth function examples

  • Hand written note

  • Past videos (2020): (PDF) Growth Function (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    
    

Week 11, Apr 6 - Apr 8, 2021

• Apr 6, 2021. Lecture 21. VC dimension

  • Hand written note

  • Additional note

  • Past videos (2020): (PDF) VC Dimension (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    

• Apr 8, 2021. Lecture 22. Bias and Variance

  • Hand written note

  • Past videos (2020): (PDF) Bias and Variance (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    
    

Week 12, Apr 13 - Apr 15, 2021

• Apr 13, 2021. No class

  • University reading day

• Apr 15, 2021. Lecture 23. Overfitting

  • Hand written note

  • Past videos (2020): (PDF) Overfit (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)
    
    

Week 13, Apr 20 - Apr 22, 2021

• Apr 20, 2021. Lecture 24. Into to neural network

  • Hand written note

  • Past videos (2020): (PDF) Feature Extraction 2: Hand-Crafted and Deep Features (Video 1) (Video 2) (Video 3)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for MNIST dataset (URL)
    

• Apr 22, 2021. Lecture 25. Convolutional structures and back propagation

  • Hand written note

  • Psat videos (2020): (PDF) Multi-Layer Perceptron and Back Propagation (Video 1) (Video 2)
    

  • Read the references in reading list page of the slides (PDF)
    
    

Week 14, Apr 27 - Apr 29, 2021

• Apr 27, 2021. Lecture 25. Generative adversarial networks

  • The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

  • Tutorial about GAN

  • Demo for GAN

  • Detecting Photoshopped Faces by Scripting Photoshop

• Apr 29, 2021. Lecture 26. Conclusion

  • capsule network (PDF) (Video)

  • (PDF) Conclusion: Practical Advices (Video 1) (Video 2)

  • How to write a good paper

  • What courses to take
    
    

Tutorials

• Tutorial 0 Python Basics (PDF)

• Tutorial 1 Linear Algebra Review (PDF)

• Tutorial 2 Probability Review (PDF)

• Tutorial 3 Linear Regression Examples (PDF)

• Tutorial 4 Optimization Review (PDF)

• Tutorial 5 Python for Image Processing (URL) (Password is in Brightspace)

• Tutorial 6 Python for Matrices (URL) (Password is in Brightspace)

• Tutorial 7 Python for Plotting (URL) (Password is in Brightspace)