Notice: For the latest information and guidance on Purdue's response to COVID-19 please visit:

Generative Models


Credit Hours:


Start Date:

January 19, 2021

Learning Objective:


Generative models are a powerful alternative to discriminative models that, when properly specified, estimate their parameters more efficiently and can generate samples from the distribution of their input data, but also can be used (like discriminative models) to infer features or labels from their inputs. However, the generative and inferential faculties typically come at each other's expense. This course will cover five different attempts at finessing this trade-off, and the resulting learning algorithms: exact inference in directed graphical models (EM algorithm); sampling-based methods; deterministic approximate inference (variational EM); RBM-like architectures (contrastive-divergence learning); and generative adversarial networks (adversarial training).
Lecture outline:

  1. Preliminaries
  2. Supervised Learning of Generative Models
  3. Exact Inference in Directed Graphical Models
  4. Sampling-Based Inference in Directed Graphical Models
  5. Variational Inference
  6. The Exponential-Family Harmonium
  7. Generative Adversarial Networks
  8. Autoregressive models
  9. Current State of the Art

Topics Covered:

Computer engineering


Linear algebra, calculus, and basic probability theory

Applied / Theory:


Approximately 10 homework assignments


1 Project


2 Exams



Computer Requirements:

Other Requirements:

Recommended reference: Bishop, Christopher M. Pattern Recognition and Machine Learning. New York: Springer, 2006. Print. Information Science and Statistics. ISBN: 9780387310732

ProEd Minimum Requirements:


Tuition & Fees: