ECE 56300 - Programming Parallel Machines

Course Details

Lecture Hours: 3 Credits: 3

Areas of Specialization:

  • Computer Engineering

Counts as:

  • EE Elective
  • CMPE Selective - Special Content

Normally Offered:

Spring - odd years

Campus/Online:

On-campus and online

Requisites:

ECE 56500

Requisites by Topic:

Proficiency in C or Fortran; computer architecture

Catalog Description:

This course will enable you to write programs targeting parallel machines, using any of the four major parallel programming paradigms: MPI (message passing), OpenMP (for shared memory machines), Pthreads thread programming (for shared memory machines.) and, GPU programming (using Cuda). We will also discuss system architecture and memory and programming language coherency models, as these are necessary to develop correct parallel programs and to debug parallel programs when they are not correct. We will also spend time on sequential performance optimizations. This is not a course in parallel algorithms, although you will need implement one or more parallel algorithms for the course project.

Required Text(s):

None.

Recommended Text(s):

  1. Parallel Programming in C with MPI and OpenMP , Quinn, M. J. , McGraw-Hill Education , 2008 , ISBN No. 0071232656

Learning Outcomes

A student who successfully fulfills the course requirements will have demonstrated an ability to:

  • Write a parallel program using MPI
  • Write a parallel program using OpenMP
  • Write a parallel program using explicit threads
  • Write a GPU program using Cuda
  • Compute performance, efficiency, and performance of a parallel program
  • Decide on the suitability of a parallel algorithm for a particular programming model

Lecture Outline:

Major Topics
1 Introduction to parallelism and the course
2 Shared and distributed memory architectures, multicores, dependence, and the relationship to parallelism
3 Hardware and software coherence in shared memory systems
4 Sequential optimizations
5 OpenMP and shared memory programming
6 Pthreads, Java and shared memory programming
7 MPI and distributed memory programming
8 GPU architecture and programming
9 Tuning applications and speedup theory, Amdahl's law, strong and weak scaling, etc. theory
10 Algorithms and techniques for fast reductions, recurrences, parallel prefix, divide and conquer, super linear speedup, etc.
11 Parallelizing compilers and their limitations
12 New programming models: some of Cilk, Stream, UPC, Galois, X-10

Assessment Method:

Exams, homework, project. (3/2022)