ECE 50024: Lecture Videos

I recorded a new set of videos in Spring 2023 for distant-learning students. They are exclusively available to students enrolled in the course. Please check Brightspace for those videos. The videos below are recorded in Spring 2020. They should give a rough idea of how the course was like.

Part 1 Mathematical Background

Objective: Understand components of a machine learning pipeline through the case study of linear regression

• Lectire 00 (PDF) Introduction
  

• Lecture 01 (PDF) Linear Regression 1: Concepts and Geometry (Video 1) (Video 2)
  

• Lecture 02 (PDF) Linear Regression 2: Ridge and LASSO Regularization (Video 1) (Video 2)
  

• Lecture 03 (PDF) Linear Regression 3: Nonlinear transform, Kernel trick (Video 1) (Video 2)
  

• Lecture 04 (PDF) Optimization 1: Optimality, Convexity, and Constraints (Video 1) (Video 2) (Video 3)
  

• Lecture 05 (PDF) Optimization 2: Gradient Descent and Stochastic Gradient Descent (Video 1) (Video 2)
  

Part 2 Supervised Learning

Objective: Understand the principles behind commonly used supervised learning methods

• Lecture 06 (PDF) Linear Separability (Video 1) (Video 2) (Video 3)
  

• Lecture 07 (PDF) Feature Extraction 1: Principal Component Analysis (Video 1) (Video 2)
  

• Lecture 08 (PDF) Feature Extraction 2: Hand-Crafted and Deep Features (Video 1) (Video 2) (Video 3)
  

• Lecture 09 (PDF) Generative Method 1: Bayesian Decision Rule (Video 1) (Video 2) (Video 3)
  

• Lecture 10 (PDF) Generative Method 2: Minimum Probility of Error Rule (Video)
  

• Lecture 11 (PDF) Generative Method 3: Estimating Parameters (Video 1) (Video 2)
  

• Lecture 12 (PDF) Generative Method 4: Bayesian Priors (Video 1) (Video 2)
  

• Lecture 13 (PDF) Generative Method 5: Connecting Bayesian Decisions with Linear Regression (Video 1) (Video 2)
  

• Lecture 14 (PDF) Logistic Regression 1: Loss and Convexity (Video 1) (Video 2) (Video 3)
  

• Lecture 15 (PDF) Logistic Regression 2: Algorithms and Interpretations (Video 1) (Video 2) (Video 3)
  

• Lecture 16 (PDF) Perceptron 1: Definitions and Concepts (Video 1) (Video 2)
  

• Lecture 17 (PDF) Perceptron 2: Algorithm and Prooperties (Video 1) (Video 2) (Video 3)
  

• Lecture 18 (PDF) Multi-Layer Perceptron and Back Propagation (Video 1) (Video 2)
  

• Lecture 19 (PDF) Support Vector Machine 1: Introduction (Video 1) (Video 2)
  

• Lecture 20 (PDF) Support Vector Machine 2: Duality (Video 1) (Video 2)
  

• Lecture 21 (PDF) Support Vector Machine 3: Soft SVM and Kernel Trick (Video 1) (Video 2)
  
  

Part 3: Learning Theory

Objective: Understand the theoretical limits of machine learning algorithms

• Lecture 22 (PDF) Is Learning Feasible? (Video 1) (Video 2) (Video 3)
  

• Lecture 23 (PDF) Probability Inequality (Video 1) (Video 2) (Video 3)
  

• Lecture 24 (PDF) Probably Approximately Correct (Video 1) (Video 2)
  

• Lecture 25 (PDF) Generalization (Video 1) (Video 2)
  

• Lecture 26 (PDF) Growth Function (Video 1) (Video 2)
  

• Lecture 27 (PDF) VC Dimension (Video 1) (Video 2)
  

• Lecture 28 (PDF) Sample and Model Complexity (Video 1) (Video 2)
  

• Lecture 29 (PDF) Bias and Variance (Video 1) (Video 2)
  

• Lecture 30 (PDF) Overfit (Video 1) (Video 2)
  

• Lecture 31 (PDF) Regularization (Video 1) (Video 2) (Video 3)
  

• Lecture 32 (PDF) Validation (Video 1) (Video 2) (Video 3)
  
  

Part 4: Robust Machine Learning

Objective: Understand the robustness of machine learning algorithms

• Lecture 33 (PDF) Overview of Adversarial Attacks (Video 1) (Video 2)
  

• Lecture 34 (PDF) Minimum Distance Attack (Video 1) (Video 2)
  

• Lecture 35 (PDF) Maximum Loss Attack and Regularized Attack (Video 1) (Video 2)
  

• Lecture 36 (PDF) Defending Adversarial Attacks (Video 1) (Video 2)
  

• Lecture 37 (PDF) Robustness and Accuracy Trade-Off (Video 1) (Video 2)
  
  
  

• Lecture 38 (PDF) Conclusion: Practical Advices (Video 1) (Video 2)