ECE 69500 - Deep Learning

Lecture Hours: 3 Credits: 3

Areas of Specialization(s):

Counts as:

Experimental Course Offered: Spring 2020

Graduate standing

Catalog Description:
The purpose of this course is to teach the theory and practice of deep neural networks from basic principles through state-of-the-art methods. The class will blend hands-on programming using a variety of state-of-the-art programming frameworks with theoretical treatment based on current literature. Implementation will emphasize the use of the Pytorch language and the use of dynamic computational graphs. Competency in multivariate calculus and linear algebra, some previous experience with optimization techniques along with some programming experience is required for success in the course.

Required Text(s):
  1. Notes will be distributed through a class web site..
Recommended Text(s):
  1. Recommend:.

Lecture Outline:

Week Lecture Topics
1 Overview of Deep Learning and Its Relation to Machine Learning in General
2 Object-Oriented Python for DL Software Structures
3 Basic Concept of Neural Processing for Classification and Regression
4 Introduction to Convolutional Networks
5 Static versus Dynamic Computational Graphs
6 Residual Learning with Skipped Connections
7 Recurrent Networks
8 Neural Processing of Textual Information
9 Theory of back-propagation and automated differentiation
10 Convergence of GD and SGD
11 Local Minima and Saddle-Point Problems
12 SGD using Momentum and Adaptive Gradients (Adam)
13 Equilibrium solutions and adversarial methods
14 Generative models
15 Recurrent models
16 Reinforcement learning