ECE 50024 - Machine Learning
Course Details
Lecture Hours: 3 Credits: 3
Areas of Specialization:
- Communications, Networking, Signal & Image Processing
Counts as:
- EE Elective
- CMPE Selective - Special Content
Normally Offered:
Each Spring
Campus/Online:
On-campus and online
Requisites:
ECE 20875, ECE 26400, ECE 30200, MA 26500
Requisites by Topic:
Programming (Python & C), probability, optimization, linear algebra
Catalog Description:
The goal of this class is to help students gain a deeper understanding of the mathematical intuition and connection behind a variety of machine learning methods rather than programming per se. The four clusters of topics that will be covered in this course are: 1) Mathematical Preliminaries. Matrices, vectors, Lp norm, geometry of the norms, symmetry, positive definiteness, eigen-decomposition. Unconstrained optimization, gradient descent, convex functions, Lagrange multipliers, linear least squares. Probability space, random variables, joint distributions, multi-dimensional Gaussians. 2) Linear Classifiers. Linear discriminant analysis, separating hyperplane, multi-class classification, Bayesian decision rule, geometry of Bayesian decision rule, linear regression, logistic regression, perceptron algorithms, support vector machines, nonlinear transformations. 3) Learning Theory. Bias and variance, training and testing, generalization, PAC framework, Hoeffding inequality, VC dimension. 4) Robustness. Adversarial attack, targeted and untargeted attack, minimum distance attack, maximum loss attack, regularization-based attack. Perturbation through noises. Robustness of SVM. These topics could help you understand the principles and limitations of machine learning methods, which can be generalized to various popular tools nowadays not covered in this class. If you are taking this course as your first class in machine learning, it could lay a solid mathematical foundation for you as you journey on in this field. If you already have machine learning backgrounds, the class could possibly provide you additional understanding of machine learning from a mathematical perspective.
Required Text(s):
None.
Recommended Text(s):
- Introduction to Probability for Data Science , First Edition , Chan, Stanley , Draft
- Learning from Data , 1st Edition , Abu-Mostafa, Yaser , ALM , 2012 , ISBN No. 1600490069
- Pattern Recognition and Machine Learning , Bishop, Chris , Springer , 2011 , ISBN No. 978-0387310732
Learning Outcomes
A student who successfully fulfills the course requirements will have demonstrated an ability to:
- classify data using statistical learning methods, and an understanding of the limitations of the methods.
- estimate model parameters using regression methods.
- apply optimization algorithms to achieve the statistical learning tasks.
- evaluate results generated by different machine learning algorithms, and make interpretations.
- apply machine learning algorithms to solve complex engineering problems.
Lecture Outline:
Week | Lecture Topics |
---|---|
1 | Course overview and mathematics review; linear regression |
2 | Examples of linear regression; ridge and LASSO regression |
3 | Optimization |
4 | Linear separability |
5 | Bayesian classifier; classification error and ROC curves |
6 | Parameter estimation; logistic regression |
7 | Logistic regression; kernel trick |
8 | Intro to neural networks; convolutional structures and back propagation |
9 | Convolutional structures; recurrent networks and transformers |
10 | Attention and transformers; adversarial attacks |
11 | Probability inequality; Is learning feasible? |
12 | Probability-approximately correct; generalization bound |
13 | Growth function and VC dimension; sample and model complexity |
14 | Bias and variance; overfitting |
15 | Regularization and validation; validation and conclusion |
Assessment Method:
The grade for this course will be determined by homework, quizzes, and a final project. (11/22)