STAT 695 / ECE 695: Sparse Modeling and Algorithms in Statistical Learning

Professor Stanley H. Chan, Purdue University, Spring 2018

Announcement

  • 2/2 Syllabus updated. Homework schedule updated.

  • 1/29 Prof Chan's travel schedule is changed. We will have class.

  • Welcome to STAT 695 / ECE 695!

Course Information

Lecture: MWF 1:30-2:20
Room: TME 1012

Instructor: Professor Stanley H. Chan
Room: MSEE 338
Email: stanchan AT purdue DOT edu
Office Hour: By appointment

Syllabus: Download (PDF) (last update: 1/7/2017)
Schedule: Download (PDF) (last update: 1/7/2017)

Lecture Note

Lecutre 00 Introduction
Lecutre 01 Sparse Modeling I: LASSO, least squares, shrinkage, cross validation, uniqueness (Hastie Ch 2)
Lecutre 02 Sparse Modeling II: Logistic regression, Poisson regression, support vector machine (Hastie Ch 3)
Lecutre 03 Sparse Modeling III: Elastic Net, Group LASSO, Total Variation (Hastie Ch 4)
Lecutre 04 Sparse Theory I: Uncertainty Principle, Mutual Coherence, Spark, Uniqueness (Elad Ch 2)
Lecutre 05 Sparse Theory II: Orthogonal Matching Pursuit (OMP), MP, Weak MP, LS-OMP, Thresholding (Elad Ch 3)
Lecutre 06 Sparse Theory III: OMP, Thresholding, Basis Pursuit, Stability (Elad Ch 4 and Ch 5)
Lecutre 07 Unconstrained Optimization Part I: Optimality, Gradient Method, Descent Direction, Line Search, Convergence
Lecutre 08 Unconstrained Optimization Part II: Newton Method, Quasi-Newton, Barzilai-Borwein, Gradient Projection, Proximal Gradient
Lecutre 09 Unconstrained Optimization Part III: Accelerated Gradient, Subgradient Methods
Lecutre 10 Constrained Optimization Part I: Lagrange multiplier, KKT conditions
Lecutre 11 Constrained Optimization Part II: Duality
Lecutre 12 Constrained Optimization Part III: Method of Multipliers, ADMM
Lecutre 13 Constrained Optimization Part IV: Plug-and-Play ADMM, Interior Point Method
Lecutre 14 Dictionary Learning I: KSVD
Lecutre 15 Dictionary Learning II: Super-Resolution
Lecutre 16 Matrix Completion

Reading Assignment

List of Papers
Reading 1, Due 19-Jan, 2018 (Friday). Good Samples
Reading 2, Due 26-Jan, 2018 (Friday). Good Samples
Reading 3, Due 02-Feb, 2018 (Friday). Good Samples
Reading 4, Due 16-Feb, 2018 (Friday). Good Samples
Reading 5, Due 02-Mar, 2018 (Friday). Good Samples
Reading 6, Due 23-Mar, 2018 (Friday). Good Samples
Reading 7, Due 06-Apr, 2018 (Friday).

Survey and Forms

Pre-course Survey
Audit Agreement Form

Project

LaTeX Template (Reading): (ZIP) (PDF)
LaTeX Template (Proposal): (ZIP) (PDF)
LaTeX Template (Report): (ZIP) (PDF)
Project Titles of Spring 2016 List

LaTeX Installation and Tutorial

MikTeX Installation (Youtube)
MikTeX on Windows and Ubuntu (Youtube)
MacTeX on Mac (Youtube)