Fall 2023 – ECE 302 Probabilistic Methods In Electrical And Computer Engineering – Lecture Notes

(The lecture notes for the previous semester (spring 2019) can be found in the following link. The notes will be very similar but not necessarily identical to the lecture notes in this semester. Please use the above link for your reference only.)

All lectures of Fall 2023 are recorded by BoilerCast. You can access the videos by (i) log into Brightspace; (ii) go to ECE302; (iii) go to [Course Tools]; and (iv) go to [Kaltura Media Gallery].

Week 1: PP001-012;
Supplemental: Weight assignment & simple counting; A bent-coin example;
Reading scope: Chapters 1, and 2.1.

Topics of Week 1:

· Basic concepts of probability as a weight assignment;

· The sandwich shop example; The bent-coin example; The game-show example; The ball drawing example.

· Basic set operations.

Week 2: PP013-024;
Supplemental: HW1Q6 demo;
Reading scope: Chapters 2.2, and 2.4.

Topics of Week 2:

· Basic set operations;

· Three axioms of probability;

· Inclusion-exclusion principle;

· Probability mass function (pmf);

· Probability density function (pdf);

Week 3: PP025-033;
Supplemental: Q16 and Q26 demo. Q25 demo is in p. 27.
Reading scope: Chapter 2.4.

Topics of Week 3:

· Conditional probability;

MT1 coverage is pp. 001-033, plus the calculus and summation questions given the HWs.

Week 4: PP034-041;
Supplemental: Q29, Q33, and Q37 demo; Q36 demo in p. 39; Q38 demo in p. 45; Q39 demo in p. 46.
Reading scope: Chapter 2.5.

Topics of Week 4:

· Use conditional probability to construct Weight Assignment;

o The tree method;

o The auto-fill example; the rare-disease example; the factory chip defective rate example.

o Bayes Rule;

Week 5: PP042-057;
Supplemental: Q41, and Q47 demo, a hard drive example;
Reading scope: Chapters 3.1, 3.2, 3.3, 3.4, 3.5.

Topics of Week 5:

· Independence;

o Definition based on conditional probability;

o Definition based on the product of marginal probabilities;

· Independence between three events;

· The hard drive error correcting code example;

· Discrete Random variables;

o Definition;

o Expectation (weighted average);

o Variance and standard deviation;

o The n-th moment versus the n-th central moment;

Week 6: PP058-066;
Reading scope: Chapters 3.4, 3.5.

· Important discrete random variables;

o Bernoulli;

o Binomial;

o Geometric;

o Poisson, and the connections to binomial random variables.

Week 7: PP067-083;

Illustration of CDFs for different random variables.
Reading scope: Chapters 4.1, 4.5.

· Continuous random variables;

o Probability density function;

o Expectation;

o Variance and standard deviation;

o The n-th moment and the n-th central moment;

o Important continuous random variables;

§ Uniform;

§ Exponential, and the connection to Poisson random variables;

§ Gaussian (normal);

· Unifying ways of describing a random variable:

o Cumulative Distribution Function (CDF) and its properties;

MT2 coverage is pp. 034-083, including HW6 and “how to compute CDF from pmf”.

Week 8: PP084-093;
Reading scope: Chapters 4.1, 4.5.

· Unifying ways of describing a random variable:

o Cumulative Distribution Function (CDF) and its properties;

§ Random variables of mixed type.

§ Use the CDF of a random variable X to find the CDF of a different random variable Y.

Week 9: PP094-113;
Reading scope: Chapters 4.5, 4.7.

· Unifying ways of describing a random variable:

o Cumulative Distribution Function (CDF);

§ Use CDF to find expectations;

o Generalized probability density function (pdf);

§ Random variables of mixed type.

o Characteristic functions;

§ Use characteristic functions to find the moments;

o Moment generating functions;

§ Use the moment generating functions to find the moments;

o Probability generating functions;

· Describe the conditional probability:

o Conditional pmf;

o Conditional pdf;

o Conditional cdf;

o Conditional generalized pdf;

Week 10: PP114-130;

· Functions of Random Variables:

o Use CDF to describe the distribution of the functions of random variables.

· Linear function of X;

o Expectation;

o Variance;

· Linear function of Gaussian random variables;

o Standard Gaussian random variables;

o Q functions;

o erfc functions (error function);

· Probability Bounds;

o Union Bound;

Week 11: PP131-133;

· Probability Bounds;

o Union Bound;

o Markov inequality;

o Chebyshev inequality;

Week 12: PP134-148;

· Probability Bounds;

o Chernoff Bound;

· Pairs of Random Variables;

o From marginal and conditional distributions to joint distributions;

o Independence;

o From joint distributions to marginal distributions;

o Expectations of joint random variables;

· Joint probability mass functions (joint pmf);

· Joint probability density functions (joint pdf);

MT3 coverage is pp. 084-148, including part of HW9 and HW10. The coverage is exclusively based on page numbers, not based on what is in HW9 or HW10.

Week 13: PP149-163;

· Joint probability density functions (joint pdf);

· Joint cumulative distribution functions (joint cdf), and their properties;

Weeks 14 and 15: PP164-180;

· Revisit the independence of joint random variables;

· Revisit the expectation of joint random variables;

o Properties of (joint) expectation;

o Product of expectation = expectation of product if independence.

· Revisit conditional expectation;

· (j,k)-th moment and (j,k)-th central moment;

· Correlation versus covariance;

o Orthogonal versus uncorrelated;

o The relationship versus independence;

o Correlation coefficient;

· Linear functions of 2-dim random variables;

· 2-dim joint Gaussian random variables and its illustration;

Week 16: PP181-198;
Topics of Week 16:

· Properties of 2-dim joint Gaussian random variables;

· Uncorrelated joint Gaussian random variables;

· Detection & Estimation;

o Maximum A posteriori Probability (MAP) detector;

o Maximum Likelihood (ML) detector;

Final exam coverage: Q1 to Q4: PP001-198; Q5 to Q8: PP149-198. Please use the lecture page number to determine whether the material will be covered or not.