Every AI revolution rides on the back of an HPC revolution

Today's critical challenge is how to do AI training and inference without destroying the planet with the energy requirement. Parallelism is an essential element of that. An energy optimal/efficient solution to any computing problem subsumes time efficiency (energy being the product of time and (average) power, a faster algorithm/program gives a multiplicative reduction in energy).

In this class you will learn how to make programs go as fast as possible on a number of platforms using a range of techniques. We will cover concepts of designing, writing, debugging, and analyzing parallel programs.

We will study the two main paradigms: shared memory (on CPUs and GPUs), and message passing on distributed memory machines. We will also examine a number of parallel algorithms for a range of problems and algorithm analysis using the work-span model. We will also examine a number of parallel algorithms for a range of problems.

This semester, CS475 will be combined with CS575, Parallel Processing. The lecture material will be largely shared, there may be special advanced lectures, covering material that is optional (extra credit) for students in CS475. In addition, each assignment will have such extra credit questions.