ECE 595 / STAT 598: Lecture

This schedule is for Spring 2021.

Week 1 Jan 19 - 21, 2021

• Jan 19, 2021. Lecture 0. Course overview. Mathematics review.

  • Live video recording (2021): Available in Brightspace.

  • Please review Tutorials 0, 6, 7 (See below)
    

• Jan 21, 2021. Lecture 1. Linear regression.

  • Past videos (2020): (PDF) Linear Regression Concepts and Geometry (Video 1) (Video 2)

  • See Chapter 7.1 of my book (URL)

  • Please review Tutorials 1 and 2 (See below)

  • Python demo for linear regression (URL)
    
    

Week 2, Jan 26 - 28, 2021

• Jan 26, 2021. Lecture 2. Outliers in linear regression

  • Hand writte note

  • Live video recording (2021): Available in Brightspace.

  • See Chapter 7.1 of my book (URL)

  • Please review Tutorials 3 (See below)

  • Python demo for linear regression with outliers (URL)
    

• Jan 28, 2021. Lecture 3. Ridge and LASSO Regularization

  • Hand writte note

  • Past videos (2020): (PDF) Ridge and LASSO Regularization (Video 1) (Video 2)

  • See Chapter 7.4 of my book (URL)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for Ridge and LASSO (URL)
    
    

Week 3, Feb 2 - 4, 2021

• Feb 2, 2021. Lecture 4. Optimization: Concepts

  • Hand written note

  • Past videos (2020): (PDF) Optimization 1: Optimality, Convexity, and Constraints (Video 1) (Video 2) (Video 3)

  • Read the references in reading list page of the slides (PDF)

  • Please review Tutorials 4 (See below)

  • Python demo for CVX (URL)
    

• Feb 4, 2021. Lecture 5. Optimization: Algorithms

  • Hand written note

  • Past videos (2020): (PDF) Optimization 2: Gradient Descent and Stochastic Gradient Descent (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)

  • Please review Tutorials 4 (See below)

  • Python demo for gradient descent (URL)
    
    

Week 4, Feb 9 - 11, 2021

• Feb 9, 2021. Lecture 6. Linearly Separable

  • Hand written note

  • Past videos (2020): (PDF) Linear Separability (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)

  • The proof of the separating hyperplane theorem is optional, but you are welcome to read.

  • Python demo for constrained optimization (URL)
    

• Feb 11, 2021. Lecture 7. Bayesian Classifier 1

  • Hand written note

  • Past videos (2020): (PDF) Linear Separability (Video 3)

  • Past videos (2020): (PDF) Bayesian Decision Rule (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for Bayesian decision rule (URL)
    
    

Week 5, Feb 16 - 18, 2021

• Feb 16, 2021. Lecture 8. Bayesian Classifier 2

  • Hand written note

  • Past videos (2020): (PDF) Bayesian Decision Rule (Video 2) (Video 3)

  • Past videos (2020): (PDF) Generative Method 2: Minimum Probility of Error Rule (Video) (First 30 minutes)

  • Read the references in reading list page of the slides (PDF)

  • Python demo for Bayesian decision rule (URL)
    

• Feb 18, 2021. Lecture 9. Classification Error / ROC Curve

  • Hand written note

  • Past videos (2020): (PDF) Generative Method 2: Minimum Probility of Error Rule (Video) (Second 30 minutes)

  • Read the references in reading list page of the slides (PDF)

  • See Chapter Chapter 9.5 of my book (URL)

  • Python demo for ROC (URL)
    
    

Week 6, Feb 23 - 25, 2021

• Feb 23, 2021. Lecture 10. Estimating parameters

  • Hand written note

  • Past videos (2020): (PDF) Estimating Parameters (Video 1) (Video 2)

  • Past videos (2020): (PDF) Bayesian Priors (Video 1)

  • Read the references in reading list page of the slides (PDF) and (PDF)

  • Read Chapter 8.1 and Chapter 8.3 of my book (URL)
    

• Feb 25, 2021. Lecture 11. Logistic regression 1

  • Hand written note

  • Past videos (2020): (PDF) Loss and Convexity (Video 1) (Video 2) (Video 3)

  • Read the references in reading list page of the slides (PDF) and (PDF)

  • Python demo for logistic regression (URL)
    
    

Week 7, Mar 2 - Mar 4, 2021

• Mar 2, 2021. Lecture 12. Logistic regression 2

  • Hand written note

  • Past videos (2020): (PDF) Algorithms and Interpretations (Video 1) (Video 2)

  • Read the references in reading list page of the slides (PDF) and (PDF)

  • Python demo for logistic regression (URL)
    
    

• Mar 4, 2021. Lecture 13. Kernel trick

  • Hand written note

  • Past videos (2020): (PDF) Nonlinear transform, Kernel trick (Video 1) (Video 2)
    

  • Read the references in reading list page of the slides (PDF)

  • Python demo for logistic regression (URL)
    
    

Tutorials

• Tutorial 0 Python Basics (PDF)

• Tutorial 1 Linear Algebra Review (PDF)

• Tutorial 2 Probability Review (PDF)

• Tutorial 3 Linear Regression Examples (PDF)

• Tutorial 4 Optimization Review (PDF)

• Tutorial 5 Python for Image Processing (URL) (Password is in Brightspace)

• Tutorial 6 Python for Matrices (URL) (Password is in Brightspace)

• Tutorial 7 Python for Plotting (URL) (Password is in Brightspace)