STAT 695 / ECE 695: Sparse Modeling and Algorithms in Statistical Learning

Announcement

Course Information

Lecture: MWF 1:30-2:20
Room: TME 1012

Instructor: Professor Stanley H. Chan
Room: MSEE 338
Email: stanchan AT purdue DOT edu
Office Hour: By appointment

Syllabus: Download (PDF) (last update: 1/7/2017)

Schedule: Download (PDF) (last update: 1/7/2017)

Lecture Note

Lecutre 00 Introduction
Lecutre 01 Sparse Modeling I: LASSO, least squares, shrinkage, cross validation, uniqueness (Hastie Ch 2)
Lecutre 02 Sparse Modeling II: Logistic regression, Poisson regression, support vector machine (Hastie Ch 3)
Lecutre 03 Sparse Modeling III: Elastic Net, Group LASSO, Total Variation (Hastie Ch 4)
Lecutre 04 Sparse Theory I: Uncertainty Principle, Mutual Coherence, Spark, Uniqueness (Elad Ch 2)
Lecutre 05 Sparse Theory II: Orthogonal Matching Pursuit (OMP), MP, Weak MP, LS-OMP, Thresholding (Elad Ch 3)
Lecutre 06 Sparse Theory III: OMP, Thresholding, Basis Pursuit, Stability (Elad Ch 4 and Ch 5)
Lecutre 07 Unconstrained Optimization Part I: Optimality, Gradient Method, Descent Direction, Line Search, Convergence
Lecutre 08 Unconstrained Optimization Part II: Newton Method, Quasi-Newton, Barzilai-Borwein, Gradient Projection, Proximal Gradient
Lecutre 09 Unconstrained Optimization Part III: Accelerated Gradient, Subgradient Methods
Lecutre 10 Constrained Optimization Part I: Lagrange multiplier, KKT conditions
Lecutre 11 Constrained Optimization Part II: Duality
Lecutre 12 Constrained Optimization Part III: Method of Multipliers, ADMM
Lecutre 13 Constrained Optimization Part IV: Plug-and-Play ADMM, Interior Point Method
Lecutre 14 Dictionary Learning I: KSVD
Lecutre 15 Dictionary Learning II: Super-Resolution
Lecutre 16 Matrix Completion

Reading Assignment

List of Papers

Reading 1, Due 19-Jan, 2018 (Friday). Good Samples
Reading 2, Due 26-Jan, 2018 (Friday). Good Samples
Reading 3, Due 02-Feb, 2018 (Friday). Good Samples
Reading 4, Due 16-Feb, 2018 (Friday). Good Samples
Reading 5, Due 02-Mar, 2018 (Friday). Good Samples
Reading 6, Due 23-Mar, 2018 (Friday). Good Samples
Reading 7, Due 06-Apr, 2018 (Friday).

Survey and Forms

Pre-course Survey
Audit Agreement Form

Project

LaTeX Template (Reading): (ZIP) (PDF)

LaTeX Template (Proposal): (ZIP) (PDF)

LaTeX Template (Report): (ZIP) (PDF)

Project Titles of Spring 2016 List

LaTeX Installation and Tutorial

MikTeX Installation (Youtube)

MikTeX on Windows and Ubuntu (Youtube)

MacTeX on Mac (Youtube)