ECE 49595 - Introduction to Reinforcement Learning

Note:

Students are allowed to count this course OR ECE 59500 Reinforcement Learning: Theory and Algorithms toward ECE credits. If both are taken, one will count as a Complementary Elective.

Course Details

Lecture Hours: 3 Credits: 3

Counts as:

  • EE Elective
  • CMPE Selective - Special Content

Experimental Course Offered:

Spring 2024

Campus/Online:

On-campus only

Requisites:

ECE 30200 and ECE 20875 and MA 26100 and MA 26500

Requisites by Topic:

Undergraduate understanding of probability, programming, multivariate calculus and linear algebra

Catalog Description:

This course introduces the foundations and the recent advances of reinforcement learning, an area of machine learning closely tied to optimal control that studies sequential decision-making under uncertainty. It aims to create a deep understanding of the theoretical and algorithmic foundations of reinforcement learning while discussing the practical considerations and various extensions of reinforcement learning.

Required Text(s):

None.

Recommended Text(s):

  1. Dynamic Programming and Optimal Control , 4th Edition , Dimitri P. Bertsekas , Athena Scientific , 2011 , ISBN No. 13: 978-1886529441
  2. Foundations of Deep Reinforcement Learning , 1st Edition , Laura Graesser and Wah Loon Keng , Addison-Wesley Professional , 2019 , ISBN No. 13: 978-0135172384
  3. Reinforcement Learning: An Introduction , 2nd Edition , Richard S. Stton and Andrew G. Barto , MIT Press , 2018 , ISBN No. 13: 978-0262039246

Learning Outcomes:

A student who successfully fulfills the course requirements will have demonstrated:
  1. an understanding of different problem formulations for reinforcement learning. [1]
  2. an ability to apply various algorithmic solutions to a wide range of sequential decisionmaking problems. [1]
  3. an ability to simulate data-driven sequential decision-making algorithms and analyze their performance. [6]

Lecture Outline:

Week(s) Lecture Topics
1-5 Fundamentals: Markov Decision Processes, Dynamic Programming, value iteration and policy iteration, model-free methods, and on-policy and off-policy methods
6-14 Learning problems and Algorithms: Background on deep learning, deep reinforcement learning methods, policy gradient methods, actor-critic methods, model-based reinforcement learning, models with continuous states and actions, inverse reinforcement learning, multi-agent reinforcement learning, reinforcement learning with partial observability
15 Case Studies

Assessment Method:

Homework and exams