ECE 645: Estimation Theory

Professor Stanley H. Chan, Purdue University, Spring 2015

Announcement

03/23/2015 Schedule of Lecture Note updated. Please check. Thanks for the volunteer.
01/16/2015 Schedule of Lecture Note updated. Please check.
12/11/2014 Welcome to ECE645! This is the official course website. Please check frequently to see if there are updates.

Course Information

Instructor: Professor Stanley H. Chan
Room: MSEE 218
Email: stanleychan AT purdue DOT edu
Office Hour: Tuesday 4-5pm; Or by email appointment.

Lecture Notes

Student Lecture Note 01 Bayes Decision Theory (Lecture 1-4, by S. Chatzidakis)
Student Lecture Note 02 Neyman Pearson Test (Lecture 5-7, by J. Jeong)
Student Lecture Note 03 Composite Hypothesis Testing (Lecture 8-10, by H. Wen)
Student Lecture Note 04 Limit Theory (Lecture 11-12, by J. Li)
Student Lecture Note 05 Large Deviation Theory (Lecture 13-14, by S. Pereira)
Student Lecture Note 06 Minimum Variance Unbiased Estimator (Lecture 15-17, by B. Vondersaar)
Student Lecture Note 07 Maximum Likelihood Estimation (Lecture 18-20, by S. Fang)
Student Lecture Note 08 Properties of MLE (Lecture 21-23, by H. Wen)
Student Lecture Note 09 Bayesian Estimation (Lecture 24-27, by J. Jeong)
Student Lecture Note 10 EM Algorithm (Lecture 28-31, by S. Fang)
Student Lecture Note 11 Iterative Algorithm (Lecture 32-33, by B. Vondersaar)
Student Lecture Note 12 Kalman Filter (Lecture 34-36, by S. Chatzidakis)

Hand Written Notes

Lecture 01 2015-01-14 Review of Probability (Poor I)
Lecture 02 2015-01-16 Bayes Decision Theory (Poor II.A - II.B, Levy 2.1-2.2)
Lecture 03 2015-01-21 Binary Hypothesis Testing (Poor II.B, Levy 2.1-2.2)
Lecture 04 2015-01-23 M-ary Hypothesis Testing (Poor II.B, Levy 2.7)
Lecture 05 2015-01-26 Trade off in False Alarm and Miss (Poor II.D, Levy 2.4)
Lecture 06 2015-01-28 Neyman Pearson Testing I (Poor II.D, Levy 2.4)
Lecture 07 2015-01-30 Neyman Pearson Testing II (Poor II.D, Levy 2.4)
Lecture 07 2015-02-02 Neyman Pearson Testing II continue (Poor II.D, Levy 2.4)
Lecture 08 2015-02-04 Composite Hypothesis Testing I – Nuisance Parameter (Poor II.E, Levy 5.1)
Lecture 09 2015-02-06 Composite Hypothesis Testing II – Uniformly Most Powerful Test (Poor II.E, Levy 5.2)
Lecture 10 2015-02-09 Composite Hypothesis Testing III – LMP and GLRT (Poor II.E, Levy 5.5)
Lecture 11 2015-02-11 Law of Large Number (Bertsekas Ch.5)
Lecture 12 2015-02-13 Central Limit Theorem (Bertsekas Ch.5)
Lecture 13 2015-02-16 Large Deviation Analysis I – General procedure (Poor III.C, Levy 3.2, Additional Reading)
Lecture 14a 2015-02-18 Large Deviation Analysis II – Detection Performance (Poor III.C, Levy 3.2, Additional Reading)
Lecture 14b 2015-02-20 Large Deviation Analysis III – Detection Performance (Poor III.C, Levy 3.2, Additional Reading)
Lecture 14 Supp Supplementary Note for Large Deviation III
Lecture 15 2015-02-23 MVUE I – Sufficient Statistics (Poor IV.C)
Lecture 16 2015-02-25 MVUE II – Rao-Blackwell (Poor IV.C)
Lecture 17 2015-02-27 MVUE III – Complete Family (Poor IV.C)
Lecture 17 Supp Supplementary Note for MVUE III
Lecture 18 2015-03-02 Maximum Likelihood I (Poor IV.D)
Lecture 19 2015-03-04 Maximum Likelihood II (Fisher Information) (Poor IV.D, Kay I Ch.3)
Lecture 20 2015-03-06 Maximum Likelihood III (Cramer Rao Lower Bound) (Poor IV.D, Kay I Ch.3)
Lecture 21 2015-03-09 Efficiency of MLE (Poor IV.D)
Lecture 22 2015-03-11 Efficiency of MLE (Poor IV.D)
Mid Term 2015-03-13 Solution
Lecture 23 2015-03-23 Consistency of MLE (Poor IV.D)
Lecture 24 2015-03-25 Bayesian Estimation I (MMSE, MMAE) (Poor IV.B)
Lecture 25 2015-03-27 Bayesian Estimation II (MAP, Examples) (Poor IV.B)
Lecture 26 2015-03-30 Bayesian Estimation III (Vector and Joint Gaussian) (Poor IV.B)
Lecture 27 2015-04-01 Bayesian Estimation IV (LMMSE) (Poor IV.B)
Lecture 27 Supp Supplementary Note for Bayesian Estimation
Lecture 28 2015-04-03 EM Algorithm I (Basic concepts) (Additional Reading)
Lecture 29 2015-04-06 EM Algorithm II (Gaussian Mixtures) (Additional Reading)
Lecture 30 2015-04-08 EM Algorithm III (Bernoulli Mixtures) (Additional Reading)
Lecture 31 2015-04-10 EM Algorithm IV (Convergence and Prior) (Additional Reading)
Lecture 32 2015-04-13 Iterative Algorithm for MLE and MAP (Additional Reading) (MATLAB Code)
Lecture 33 2015-04-15 Stein Unbiased Risk Estimator (Additional Reading 1) (Additional Reading 2)
Lecture 34 2015-04-17 Kalman-Bucy Filtering I (Main results) (Poor V.B)
Lecture 35 2015-04-20 Kalman-Bucy Filtering II (Implementation and Examples) (Poor V.B)
Lecture 36 2015-04-22 Orthogonality Principles and Wiener-Hopf Equation (Poor V.C)

ECE 645: Estimation Theory

Announcement

Course Information

Lecture Notes

Hand Written Notes

Homework

Project

LaTeX Installation and Tutorial