This textbook focuses on practical parallel C++ programming at the graduate student level. In particular, it shows the APIs and related language features in the C++ 17 and C++ 20 standards, covering both single node and distributed systems. It shows that with the parallel features in the C++ 17 and C++ 20 standards, learning meta-languages like OpenMP is no longer necessary. Using the C++ standard library for parallelism and concurrency (HPX), the same language features can be extended to distributed codes, providing a higher-level C++ interface to distributed programming than the Message Passing Interface (MPI).

The book starts with the single-threaded implementation of the fractal sets, e.g. Julia set, and Mandelbrot set, using the C++ Standard Library (SL)’s container and algorithms. This code base is used for parallel implementation using low-level threads, asynchronous programming, parallel algorithms, and coroutines. The asynchronous programming examples are then extended to distributed programming using the C++ standard library for parallelism and concurrency (HPX). Octo-Tiger, an astrophysics code for stellar merger, is used as a showcase for a portable, efficient, and scalable high-performance application using HPX.

The book’s core audience is advanced undergraduate and graduate students who want to learn the basics of parallel and distributed C++ programming but are not computer science majors. Basic C++ knowledge, like functions, classes, loops, and conditional statements, is assumed as a requirement, while C++ advanced topics, like generic programming, lambda functions, smart pointers, and move semantics, are briefly summarized in the appendix.

Table of Contents

Part I Preliminaries

1 Compiling and Running the Code and Examples in This Book

Part II Introduction to C++ and C++ Standard Library

2 About C++, C++ Standard, and the C++ Standard Library

3 C++ Standard Library

4 Example Mandelbrot Set and Julia Set

Part Ill The C++ Standard Library for Concurrency and Parallelism (HPX)

5 Why HPX?

6 The C++ Standard Library for Parallelismand Concurrency (HPX)

Part IV Parallel Programming

7 Parallel Programming

8 Programming with Low Level Threads

9 Asynchronous Programming

10 Parallel Algorithms

11 Coroutines

12 Benchmarking the Fractal Set Codes

Part V Dist ributed Programming

13 Dist ributed Computing and Programming

14 Distributed Programming Using HPX

15 Examples of Dist ributed Programming

16 Some Remarks on MPl+OpenMP and HPX

Part VI A Showcase for a Portable High Performance Application Using HPX

17 Accelerator Cards

18 Octo-Tiger, a Showcase for a Portable High Performance Application

Part VII Conclusion and Outlook

19 Conclusion and Outlook

About the Authors

Patrick Diehl is a faculty member at the Department of Physics & Astronomy at Louisiana State University (LSU) and a senior research scientist at LSU's Center for Computation and Technology (CCT). His research interests are computational fracture mechanics, scientific computing, and high-performance computing. Next to research, he is devoted to teaching C++ and parallel programming to students and the community.

Steven R. Brandt is the Assistant Director for Computational Science at the Center for Computation and Technology at LSU. His research interests include parallel programming, parallel languages and tools, science gateways, and high performance computing (HPC). He has served as PI on several grants supporting the Einstein Toolkit, a widely used open source project for studying numerical relativity.

Hartmut Kaiser is a faculty member at the CS and EE departments at Louisiana State University (LSU) and a senior research scientist at LSU's Center for Computation and Technology (CCT). He is probably best known for his involvement in open source software projects, such as being the author of several C++ libraries he has contributed to Boost, which are in use by thousands of developers worldwide. His current research is focused on leading the STE||AR group at CCT working on the practical design and implementation of future execution models and programming methods.