ECE 49595 - Introduction to Reinforcement Learning

Note:

Students are discouraged from taking both this course and ECE 59500 Reinforcement Learning: Theory and Algorithms due to the amount of overlap between the courses. Should a student take both, the second will count as a Complementary Selective.

Course Details

Lecture Hours: 3 Credits: 3

Counts as:

  • EE Elective
  • EE - Automatic Control Concentration
  • EE - Artificial Intelligence and Machine Learning Concentration
  • CMPE Selective - Special Content
  • CMPE - Artificial Intelligence and Machine Learning Concentration

Experimental Course Offered:

Spring 2024, Spring 2025, Spring 2026

Campus/Online:

On-campus only

Requisites:

ECE 30200 and ECE 20875 and MA 26100 and [MA 26200 or MA 26500]

Requisites by Topic:

Undergraduate understanding of probability, programming in Python, multivariate calculus and linear algebra

Catalog Description:

This course introduces the foundations of reinforcement learning, an area of machine learning closely tied to optimal control that studies sequential decision-making under uncertainty. It provides a broad overview of different interrelated problems and algorithmic solutions in reinforcement learning and its extensions, such as deep learning-driven methods.

Required Text(s):

None.

Recommended Text(s):

  1. Dynamic Programming and Optimal Control , 4th Edition , Dimitri P. Bertsekas , Athena Scientific , 2011 , ISBN No. 13: 978-1886529441
  2. Foundations of Deep Reinforcement Learning , 1st Edition , Laura Graesser and Wah Loon Keng , Addison-Wesley Professional , 2019 , ISBN No. 13: 978-0135172384
  3. Reinforcement Learning: An Introduction , 2nd Edition , Richard S. Stton and Andrew G. Barto , MIT Press , 2018 , ISBN No. 13: 978-0262039246

Learning Outcomes:

A student who successfully fulfills the course requirements will have demonstrated:
  1. an understanding of different problem formulations for reinforcement learning. [1]
  2. an ability to apply various algorithmic solutions to a wide range of sequential decisionmaking problems. [1]
  3. an ability to simulate data-driven sequential decision-making algorithms and analyze their performance. [6]

Lecture Outline:

Week(s) Lecture Topics
1-5 Fundamentals: Markov Decision Processes, Dynamic Programming, Value Iteration and Policy Iteration, Model-Free Methods, and On-Policy and Off-Policy Methods
6-14 Learning problems and Algorithms: Background on deep learning, deep reinforcement learning methods, policy gradient methods, actor-critic methods, model-based reinforcement learning, models with continuous states and actions, reinforcement learning with partial observability, multi-agent reinforcement learning, imitation learning and reinforcement learning from human feedback.
15 Case Studies

Assessment Method:

Exams, quizzes and homework