Spring 2019 – ECE 302 Probabilistic Methods In Electrical And Computer Engineering – Lecture Notes
Skeleton of the lecture notes (for students taking
notes using iPad)
|
|
|
|
|
|
Week 1: PP001-014;
Supplemental: Weight assignment & simple counting;
A bent-coin example;
Reading scope: Chapters 1, and 2.1.
Topics of
Week 1:
·
Basic concepts of
probability as a weight assignment;
·
The sandwich shop
example; The bent-coin example; The game-show example; The ball drawing
example.
·
Basic set
operations.
Week 2: PP015-029;
Supplemental: HW1Q6 demo;
Reading scope: Chapters 2.2, and 2.4.
Topics of
Week 2:
·
Basic set
operations;
·
Three axioms of
probability;
·
Inclusion-exclusion
principle;
·
Probability mass
function (pmf);
·
Probability
density function (pdf);
·
Conditional
probability;
Week 3: PP030-039;
Supplemental: HW2Q6 and Q14 demo, an auto-fill example, a rare-disease
example.
Reading scope: Chapter 2.4.
Topics of
Week 3:
·
Conditional
probability;
·
Use conditional
probability to construct Weight Assignment;
o
The tree method;
·
The auto-fill
example; the rare-disease example; the factory chip
defective rate example.
MT1 coverage: PP001-039, and HW1 to HW3. There will be some pure computational
questions just like those in HW1 to HW3.
Week 4: PP040-045;
Supplemental: HW3Q3, Q7, and Q13 demo;
Reading scope: Chapter 2.5.
Topics of
Week 4:
·
Use conditional
probability to construct Weight Assignment;
o
The tree method;
o
Bayes
Rule;
·
Independence;
o
Definition based
on conditional probability;
o
Definition based
on the product of marginal probabilities;
Week 5: PP046-059;
Supplemental: HW4Q6, Q8, and Q10 demo, a hard drive example;
Reading scope: Chapters 3.1, 3.2, 3.3, 3.4, 3.5.
Topics of
Week 5:
·
Independence
between three events;
·
The hard drive
error correcting code example;
·
Discrete Random
variables;
o
Definition;
o
Expectation
(weighted average);
o
Variance and
standard deviation;
o
The n-th moment
versus the n-th
central moment;
o
Important
discrete random variables;
§
Bernoulli;
§
Binomial;
Week 6: PP060-078;
Supplemental: HW5Q5, Q7, and Q10 demo;
Reading scope: Chapters 4.2, 4.3, 4.4.
Topics of
Week 6:
·
Important discrete
random variables;
o
Continue from
Week 5:
§
Geometric;
§
Poisson, and the
connections to binomial random variables.
·
Continuous random
variables;
o
Probability
density function;
o
Expectation;
o
Variance and
standard deviation;
o
The n-th moment
and the n-th
central moment;
o
Important
continuous random variables;
§
Uniform;
§
Exponential, and
the connection to Poisson random variables;
Week 7: PP079-095; Illustration of CDFs
for different random variables.
Reading scope: Chapters 4.1, 4.5.
Topics of
Week 7:
·
Important
continuous random variables;
o
Continue from
Week 6:
§
Exponential, and
the memoryless property;
§
Gaussian
(normal);
·
Unifying ways of
describing a random variable:
o
Cumulative Distribution
Function (CDF) and its properties;
§
Random variables
of mixed type.
§
Use the CDF of a
random variable X to find the CDF of a different random variable Y.
MT2 coverage: PP040-089, and HW4 to HW6. Part of HW7 will be covered as well. Please use the lecture page number to
determine whether the material will be covered or not.
Week 8: PP096-118;
Supplemental: HW7Q7 demo;
Reading scope: Chapter 4.7.
Topics of
Week 8:
·
Unifying ways of
describing a random variable:
o
Cumulative
Distribution Function (CDF);
§
Use CDF to find
expectations;
o
Generalized
probability density function (pdf);
o
Characteristic
functions;
§
Use
characteristic functions to find the moments;
o
Moment generating
functions;
§
Use the moment
generating functions to find the moments;
o
Probability
generating functions;
·
Describe the
conditional probability:
o
Conditional pmf;
o
Conditional pdf;
o
Conditional cdf;
o
Conditional
generalized pdf;
Week 9: PP119-135;
Supplemental: HW8Q3 and Q4 demo;
Reading scope: Chapters 4.6. 5.1, 5.2.
Topics of
Week 9:
·
Unifying ways of
describing a random variable:
o
Probability
generating functions;
§
Use the
probability generating functions to find the moments;
·
Functions of
Random Variables:
o
Use CDF to
describe the distribution of the functions of random variables.
·
Linear function
of X;
o
Expectation;
o
Variance;
·
Linear function
of Gaussian random variables;
o
Standard Gaussian
random variables;
o
Q functions;
o
erfc functions (error function);
·
Probability
Bounds;
o
Union Bound;
o
Markov
inequality;
o
Chebyshev inequality;
o
Chernoff inequality;
A high-level outline of
all the subjects discussed in Weeks 1 to 9.
Week 10: PP136-150;
Supplemental: HW9Q6 demo;
Reading scope: Chapters 5.4, 5.5.
Topics of
Week 10:
·
Pairs of Random
Variables;
o
From marginal and
conditional distributions to joint distributions;
o
Independence;
o
From joint
distributions to marginal distributions;
o
Expectations of
joint random variables;
·
Joint probability
mass functions (joint pmf);
·
Joint probability
density functions (joint pdf);
Week 11: PP151-164;
Supplemental: HW10Q2 and Q7 demo;
Reading scope: Chapter 5.3.
Topics of
Week 11:
·
Joint probability
density functions (joint pdf);
·
Joint cumulative
distribution functions (joint cdf), and their
properties;
MT3 coverage: PP090-0161, and HW7 to HW11. Please use the lecture page number to
determine whether the material will be covered or not.
Week 12: PP165-184; Textbook illustration of 2-dimensional Gaussian random
variables.
Reading scope: Chapters 5.6, 5.7, 5.8, and 5.9.
Topics of
Week 12:
·
Revisit the
independence of joint random variables;
·
Revisit the
expectation of joint random variables;
o
Properties of
(joint) expectation;
o
Product of expectation
= expectation of product if
independence.
·
Revisit
conditional expectation;
·
(j,k)-th
moment and (j,k)-th
central moment;
·
Correlation
versus covariance;
o
Orthogonal
versus uncorrelated;
o
The
relationship versus independence;
o
Correlation
coefficient;
·
Linear
functions of 2-dim random variables;
·
2-dim
joint Gaussian random variables and its illustration;
Week 13: PP185-204;
Supplemental: HW12Q10;
Reading scope: Chapter 6.5.
Topics of
Week 13:
·
2-dim
joint Gaussian random variables;
·
Properties
of 2-dim joint Gaussian random variables;
·
Uncorrelated
joint Gaussian random variables;
·
Detection
& Estimation;
o
Maximum
A posteriori Probability (MAP) detector;
o
Maximum
Likelihood (ML) detector;
o
Minimum
Mean Square Error (MMSE) estimator;
o
Linear
MMSE estimator;
Week 14: PP205-222;
Reading scope: Chapters 6.1, 6.2, 6.3, 6.4, 7.1.
Topics of
Week 14:
·
n-dim random variables;
·
Chain
rule of the conditional probability;
·
Independence;
·
Expectation;
·
n-dimensional joint Gaussian random variables;
o
Properties
of n-dim joint Gaussian;
·
Linear
functions of n-dimensional random
variables;
·
Summation
of n independent
random variables;
o
Bernoulli
+ Bernoulli;
o
Binomial
+ binomial;
o
Poisson
+ Poisson;
o
Gaussian
+ Gaussian;
Final exam coverage: Q1 to Q4: PP001-222; Q5 to Q8: PP162-222. Please use the lecture page number to
determine whether the material will be covered or not.
Additional self-learning materials: PP223-239; A central limit
theorem illustration;
Additional
topics (Week 15):
·
Sample
Mean;
·
Weak
Law of Large Numbers;
·
Strong
Law of Large Numbers;
·
Central
Limit Theorem;
·
Random
Processes;
o
White
Gaussian random processes;
o
Mean
function;
o
Variance
function;
o
Auto-correlation
function;
o
Auto-covariance
function;
·
Wide
sense stationary (WSS) random processes;