ECE 50024: Machine Learning

Spring 2025

Date: Jan 13, 2025 - May 03, 2025

In Spring 2025, we offer the following sections, simultaneously.

• ECE 50024-002 (CRN 17000). In-person. MWF 12:30-1:20pm. Forney Hall G140.

• STAT 59800-101 (CRN 23903). In-person. MWF 12:30-1:20pm. Forney Hall G140.

• ECE 50024-003 (CRN 30467). Online (West Lafayette distance learning). Asynchronous.

• ECE 50024-EPE (CRN 29038). Online (Purdue Online). Asynchronous

Course Description

An introductory course to machine learning, with a focus on supervised learning using linear models.

• Regression. Linear regression, loss function, optimization, gradient descent, stochastic gradient descent, least squares fitting, robust regression, ridge regularization, LASSO regularization

• Linear Classifiers. Linear discriminant analysis, separating hyperplane, multi-class classification, Bayesian decision rule, geometry of Bayesian decision rule, linear regression, logistic regression, robustness of classifiers, adversarial learning

• Generative Models. Variational autoencoder, denoising diffusion probabilistic models, score matching Langevin dynamics

• Learning Theory. Bias and variance, training and testing, generalization, PAC framework, Hoeffding inequality, VC dimension.

Pre-requisites

• Purdue ECE 302 Probability

• Purdue MA 265 Linear Algebra

• Purdue ECE 647 (Optional) Convex and Stochastic Optimization and Applications

Useful Background Materials

• • Purdue ECE 302 Probability https://engineering.purdue.edu/ChanGroup/eBook.html

  • MIT 18.06 Linear Algebra: http://math.mit.edu/~gs/linearalgebra/

  • MIT 6.079 Optimization https://ocw.mit.edu/courses/6-079-introduction-to-convex-optimization-fall-2009/

  • MIT 6.012 Probability: https://www.amazon.com/Introduction-Probability-2nd-Dimitri-Bertsekas/dp/188652923X

  • Stanford EE 364A Convex Optimization: https://stanford.edu/~boyd/cvxbook/

  • Stanford ENGR 108 Linear Algebra: https://web.stanford.edu/~boyd/vmls/

Grades

In-Person Sessions (ECE 50024-002 and STAT 59800-101)

• Test (80%)

  • There will be four tests. All tests are independent and not cumulative.

  • Test 1 (20%): 45-minute test during lecture. Topics for Week 1 - Week 4.

  • Test 2 (20%): 45-minute test during lecture. Topics for Week 5 - Week 8.

  • Test 3 (20%): 45-minute test during lecture. Topics for Week 9 - Week 12.

  • Test 4 (20%): 45-minute test during lecture. Topics for Week 13 - Week 16.

  • All tests are closed-book, closed-note.

• Random Quiz (20%)

  • Throughout the semester, for attendence and quick assessment.

  • Roughly once a week or once every two weeks.

  • Will take place randomly.

  • No more than 10 minutes each.

  • Most questions are short, simple, easy.

  • We may subsitute some of the quizzes with in-class activities. Stay tuned.

• Homework (0%)

  • Homework will be assigned but not collected.

  • Solution will be distributed.

• Final Exam (0%)

  • There is no final exam.

• Project (0%)

  • There is no project.

• Curve

  • We will curve the class.

  • Undergrad and grad will use the same curve. (This is required by the undergrad and grad committee, because this is a 500-level class.)

  • Roughly speaking:

    • A: above mean + 1 std deviation

    • B: mean + 1 std deviation

    • C: mean +/- 0.5 std deviation

    • D: mean - 1 std deviation

    • F: below mean - 1 std deviation

Asynchronous Sessions (ECE 50024-003 and ECE 50024-EPE)

• Test (80%). Same as in-person classes. But the 45-minute test will be on gradescope. Protored by Examity.

• Quiz (20%). Weekly quiz on gradescope. 10 minutes each. Window: Saturday morning 8:01am ET to Monday morning 7:59am ET.

• Curve: We will curve the asynchronous sessions together with the in-class sessions. That is, the entire class only has one population and one curve.

Textbook and References

There is no official textbook for this course. Please refer to the lecture note section of the website for our lecture materials.

A few good reference books for this course are:

• Introduction to Probability for Data Science, by Stanley Chan, draft. 1st edition. 2020.

• Pattern Classification, by Duda, Hart and Stork, Wiley-Interscience; 2 edition, 2000.

• Learning from Data, by Abu-Mostafa, Magdon-Ismail and Lin, AMLBook, 2012.

• Elements of Statistical Learning, by Hastie, Tibshirani and Friedman, Springer, 2 edition, 2009.

• Pattern Recognition and Machine Learning, by Bishop, Springer, 2006.
  
  
  

Programming

We will be primarily using Python. As such, I expect you to have elementary programming skills, e.g., writing a hello world program. More information and resources on how to use Python can be found in the programming section of this website. I found Google Colab a fairly easy-to-use platform for Python programming. You can check this out.

Besides Python, we use optimization packages to solve optimization problems. Of particular importance is CVX.