Recommended Books
Here are the books that helped me out the most when learning MPI. Most of these are written by the primary designers of the Message Passing Interface. Some also include tutorials on how to use OpenMP with MPI.
Disclaimer - The last book is primarily a compilation of these tutorials by the original mpitutorial.com author (Wes Kendall).
Parallel Programming with MPI
My personal favorite MPI book. The book gives a good overview of parallel computing before delving into all the various topics of MPI programming. It goes into detail about almost every essential MPI routine, and then it provides examples of parallel programs such as matrix multiplication and sorting. The end of the book discusses libraries and debugging routines. Click here for more info.
Using MPI - 3rd Edition and Using Advanced MPI - 1st Edition
This is a more up-to-date book than the previous. The “regular” book covers the fundamentals of MPI and the “advnaced” book covers additional topics. The table of contents can be found on this website. This is a must have for advanced MPI development. Click here for Using MPI. The “Using Advanced MPI” book is currently out of print.
Parallel Programming in C with MPI and OpenMP
This book is a bit older than the others, but it is still a classic. One strong point of this book is the huge amount of parallel programming examples, along with its focus on MPI and OpenMP. Many parallel programs are discussed in great detail, including matrix multiplication, fast fourier transforms, sorting, and combinatorial searching. Click here for more info.
MPI: The Complete Reference
A complete reference guide to MPI one and two. The book does not necessarily teach MPI, but it provides a great reference and complete descriptions of every single function. One advantage of this book is that it includes Fortran routines as well as C routines. Click here for more info.
Beginning MPI (An Introduction in C)
This book is a compilation of all of the beginner tutorials on this site. It goes over everything from installing MPI on an Amazon EC2 cluster to the basics of sending and receiving with MPI to performing collective operations and reductions. If you have enjoyed the tutorials on this site and wish to have a copy of it in book format, then click here for more info.
