ECE 69500 - Sparse Representations and Signal Recovery
Note:
Assessment: Project Proposal 20%; Final Report 40%; Final Presentation 40%
Course Details
Lecture Hours: 3 Credits: 3
Areas of Specialization:
Counts as:
Experimental Course Offered:
Spring 2016
Requisites:
Sparse representation is a corner stone in statistics and engineering. The heart of model, however, is a simple and long lasting mathematical problem: Given a full rank matrix $A \in \R^{n \times m}$ with $n << m$, how do we find a solution with fewest number of non-zeros in an underdetermined system $Ax = b$? Interestingly, the answer to this question has now become the core of many problems: regression, prediction, denoising, restoration, interpolation and extrapolation, compression, sampling, detection, recognition, etc. In this course, we will explore the fundamentals of the theory behind sparse representations. We will study the uniqueness of sparse solutions, pursuit algorithms, convex relaxations, and analyze the performance. Applications in signal and image processing will be discussed.
Requisites by Topic:
undergraduate linear algebra and probability
Catalog Description:
Required Text(s):
None.
Recommended Text(s):
- Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing , Michael Elad , Springer , 2010 , ISBN No. 978-1-4419-7011-4
Lecture Outline:
| Weeks | Lecture Topics |
|---|---|
| 1-2 | Prologue: Under-determined system, Convexity, $L_0$, Sparse solution Uniqueness and Uncertainty: Uncertainty principle, Uniqueness via Spark, Mutual coherence |
| 3-4 | Pursuit Algorithms (Algorithm): Greedy algorithm, Convex relaxation,Orthogonal Matching Pursuit, Basis Pursuit Pursuit Algorithms (Guarantees): Performance analysis for Orthogonal matching pursuit; Performance analysis for basis pursuit |
| 5-7 | From Exact to Approximate Solutions: Stability of approximated solutions, Restricted Isometry, Iterated reweighted least-squares, BP Denoising Iterative Shrinkage: Proximal operator, Coordinate descent, Average performance analysis |
| 8-9 | Sparse Model for Signals: Priors and transforms of signals; Sparse-land model; analysis vs synthesis modeling Dictionary learning: the K-SVD algorithm |
| 10-12 | Applications in signal and image processing: Denoising, deblurring, compression |
| 13-15 | Advanced topics from recent literature |
Assessment Method:
Assessment: Project Proposal 20%; Final Report 40%; Final Presentation 40%