Programming Parallel Machines


Credit Hours:


Learning Objective:

After taking this course, a student should be able to (1) Having a basic understanding of computer architecture and how it affects parallel program performance, (2) write a parallel program using OpenMP, Pthreads, MPI and Cuda, (3) Parallelize a program using the the tools of (2) to run in parallel on a multicore machine, a distributed memory multi-node machine and an accelerator, (3) Computer the efficiency and performance of a parallel program and measure its scalability.


This course will allow students to do parallel computations in a variety of engineering fields and the data sciences by presenting methods and techniques for programming parallel computers, such as multicore and high-end parallel architectures. A short introduction to computer architecture and general parallelization concepts will be presented. Parallel architectures to be considered are shared-memory and distributed-memory multiprocessor systems and accelerators such as Graphics Processing Units (GPUs). Programming paradigms for these machines will be compared, including directive-based (OpenMP), thread-based (Posix threads), message passing (MPI) and a language targeting accelerators. Methodologies for analyzing and improving the performance of parallel programs will be discussed. There will be a class project in which each student develops and tunes a parallel application.

Topics Covered:

Basic concepts of parallelism, and overview sequential and parallel computer architecture, Shared memory programming with OpenMP and Pthreads, Distributed memory programming using MPI, Accellerator programming using Cuda, Speedup theory including speedup, efficiency, Amdahl???s Law, Karp-Flatt, Gustafson-Barsis, Isoefficiency, Span, Work and Speedup.


Competency in C, C++ or Fortran.

Applied / Theory:

85 / 15

Web Address:


The project will consist of programming a parallel implementation of an algorithm and analyzing its performance. There will be a default project that can be done, or students are free to propose their own project, which can be related to their job.


There will be one in-class exam.


Official textbook information is now listed in the Schedule of Classes. NOTE: Textbook information is subject to be changed at any time at the discretion of the faculty member. If you have questions or concerns please contact the academic department.

Computer Requirements:

Students will use the Purdue Scholar cluster for programming assignments. If students have access to multi-node parallel machines in their lab or job, they may do the programming on those machines as well. Students will need access to a computer to allow them to remotely login to the Purdue Scholar cluster. Piazza

ProEd Minimum Requirements:


Tuition & Fees: