STAT 695 / ECE 695: Sparse Modeling and Algorithms in Statistical Learning

Professor Stanley H. Chan, Purdue University, Spring 2018

Lecture: MWF 1:30pm - 2:20pm
Room: TME 1012
Instructor: Professor Stanley H. Chan
- Room: MSEE 338
- Email: stanchan@purdue.edu
- Office Hour: By appointment.
Syllabus: Download Syllabus PDF (Last updated: 1/7/2017)
Schedule: Download Schedule PDF (Last updated: 1/7/2017)

Lecture 00: Introduction
Lecture 01: Sparse Modeling I: LASSO, least squares, shrinkage, cross validation, uniqueness (Hastie Ch 2)
Lecture 02: Sparse Modeling II: Logistic regression, Poisson regression, support vector machine (Hastie Ch 3)
Lecture 03: Sparse Modeling III: Elastic Net, Group LASSO, Total Variation (Hastie Ch 4)
Lecture 04: Sparse Theory I: Uncertainty Principle, Mutual Coherence, Spark, Uniqueness (Elad Ch 2)
Lecture 05: Sparse Theory II: Orthogonal Matching Pursuit (OMP), MP, Weak MP, LS-OMP, Thresholding (Elad Ch 3)
Lecture 06: Sparse Theory III: OMP, Thresholding, Basis Pursuit, Stability (Elad Ch 4 & 5)
Lecture 07: Unconstrained Optimization Part I: Optimality, Gradient Method, Descent Direction, Line Search, Convergence
Lecture 08: Unconstrained Optimization Part II: Newton Method, Quasi-Newton, Barzilai-Borwein, Gradient Projection, Proximal Gradient
Lecture 09: Unconstrained Optimization Part III: Accelerated Gradient, Subgradient Methods
Lecture 10: Constrained Optimization Part I: Lagrange multiplier, KKT conditions
Lecture 11: Constrained Optimization Part II: Duality
Lecture 12: Constrained Optimization Part III: Method of Multipliers, ADMM
Lecture 13: Constrained Optimization Part IV: Plug-and-Play ADMM, Interior Point Method
Lecture 14: Dictionary Learning I: KSVD
Lecture 15: Dictionary Learning II: Super-Resolution
Lecture 16: Matrix Completion