ECE 56300 - Programming Parallel Machines

Lecture Hours: 3 Credits: 3

Counts as:
CMPE Special Content Elective
EE Elective

Normally Offered: Spring - odd years

Requisites:
ECE 56500

Catalog Description:
This course will allow students to do parallel computations in a variety of engineering fields and the data sciences by presenting methods and techniques for programming parallel computers, such as multicore and high-end parallel architectures. A short introduction to computer architecture and general parallelization concepts will be presented. Parallel architectures to be considered are shared-memory and distributed-memory multiprocessor systems and accelerators such as Graphics Processing Units (GPUs). Programming paradigms for these machines will be compared, including directive-based (OpenMP), thread-based (Posix threads), message passing (MPI) and a language targeting accelerators. Methodologies for analyzing and improving the performance of parallel programs will be discussed. There will be a class project in which each student develops and tunes a parallel application.

Supplementary Information:
Course Format: class: three lectures per week. In part, the lectures will be held in computer labs, where the students will do hands-on assignments. Exams: One in class written exam and a final written or oral exam. Computer Languages Used: C and some Fortran. Computer facilities Required: For documentation, programming, and implementation experiments, personal computers and several parallel machines will be used. Class Projects Each student will do an application project. An application of the student's choice will be converted to a parallel program and executed on a parallel machine. The project report will document the original and tuned performance for each program section as well as intermediate results of each program tuning step. Important findings of the projects will be presented in class.

Required Text(s): None.

Recommended Text(s):
  1. Research papers and course handouts.

Learning Outcomes:

A student who successfully fulfills the course requirements will have demonstrated:
  1. an understanding of the basic properties of parallel computer architectures and their relationship to parallel program design. [4]
  2. an ability to analyze a program for parallelism and express this parallelism for both shared-memory and distributed-memory machines. [1,2,6]
  3. an understanding of parallel models and programming constructs for OpenMP, MPI, and Posix threads. [4]
  4. an understanding of performance factors of parallel programs and their relationship to application characteristics and parallel programming constructs. [1,6]
  5. an ability to use parallelizing compilers to parallelize and tune the performance of application programs. [1,4,6]

Lecture Outline:

Week(s) Topics
1 Introduction and Motivation
1 Efficiency and Speedup Measures
1 Automatic Parallelization
1 Tuning Automatically Parallelized Programs
2 Explicit Program Parallelization
1 Open MP
1 MPI
1 Pthreads
2 Programming Methodologies and Tools
2 Application Studies
2 Project Discussions

Engineering Design Content:

Establishment of Objectives and Criteria
Synthesis
Analysis
Construction
Testing
Evaluation

Engineering Design Consideration(s):

Economic
Manufacturability