ECE 56300 - Programming Parallel Machines
Course Details
Lecture Hours: 3 Credits: 3
Areas of Specialization:
- Computer Engineering
Counts as:
- EE Elective
- CMPE Selective - Special Content
Normally Offered:
Spring - odd years
Campus/Online:
On-campus and online
Requisites:
ECE 56500
Requisites by Topic:
Proficiency in C or Fortran; computer architecture
Catalog Description:
This course will enable you to write programs targeting parallel machines, using any of the four major parallel programming paradigms: MPI (message passing), OpenMP (for shared memory machines), Pthreads thread programming (for shared memory machines.) and, GPU programming (using Cuda). We will also discuss system architecture and memory and programming language coherency models, as these are necessary to develop correct parallel programs and to debug parallel programs when they are not correct. We will also spend time on sequential performance optimizations. This is not a course in parallel algorithms, although you will need implement one or more parallel algorithms for the course project.
Required Text(s):
None.
Recommended Text(s):
- Parallel Programming in C with MPI and OpenMP , Quinn, M. J. , McGraw-Hill Education , 2008 , ISBN No. 0071232656
Learning Outcomes
A student who successfully fulfills the course requirements will have demonstrated an ability to:
- Write a parallel program using MPI
- Write a parallel program using OpenMP
- Write a parallel program using explicit threads
- Write a GPU program using Cuda
- Compute performance, efficiency, and performance of a parallel program
- Decide on the suitability of a parallel algorithm for a particular programming model
Lecture Outline:
Major Topics | |
---|---|
1 | Introduction to parallelism and the course |
2 | Shared and distributed memory architectures, multicores, dependence, and the relationship to parallelism |
3 | Hardware and software coherence in shared memory systems |
4 | Sequential optimizations |
5 | OpenMP and shared memory programming |
6 | Pthreads, Java and shared memory programming |
7 | MPI and distributed memory programming |
8 | GPU architecture and programming |
9 | Tuning applications and speedup theory, Amdahl's law, strong and weak scaling, etc. theory |
10 | Algorithms and techniques for fast reductions, recurrences, parallel prefix, divide and conquer, super linear speedup, etc. |
11 | Parallelizing compilers and their limitations |
12 | New programming models: some of Cilk, Stream, UPC, Galois, X-10 |
Assessment Method:
Exams, homework, project. (3/2022)