Performing Parallel Rank with MPI

Posted by & filed under Beginner MPI.

In the previous lesson, we went over MPI_Scatter, MPI_Gather, and MPI_Allgather. We are going to expand on basic collectives in this lesson by coding a useful function for your MPI toolkit – parallel rank. The code for this tutorial is available as a tgz file or can be viewed/cloned on GitHub. Recommended books View all… Read more »

MPI Reduce and Allreduce

Posted by & filed under Beginner MPI.

In the previous lesson, we went over an application example of using MPI_Scatter and MPI_Gather to perform parallel rank computation with MPI. We are going to expand on collective communication routines even more in this lesson by going over MPI_Reduce and MPI_Allreduce. The code for this tutorial is available as a tgz file or can… Read more »

Launching an Amazon EC2 MPI Cluster

Posted by & filed under Beginner MPI.

In the previous lesson, we discussed how to install MPICH2, an implementation of MPI, onto a single machine and verify that it was installed properly. Unfortunately, a single machine does not always provide a realistic environment on which to learn MPI and execute programs. Along with that, many beginners do not have immediate access to… Read more »

MPI Scatter, Gather, and Allgather

Posted by & filed under Beginner MPI.

In the previous lesson, we went over the essentials of collective communication. We covered the most basic collective communication routine – MPI_Bcast. In this lesson, we are going to expand on collective communication routines by going over two very important routines – MPI_Scatter and MPI_Gather. We will also cover a variant of MPI_Gather, known as… Read more »

MPI Broadcast and Collective Communication

Posted by & filed under Beginner MPI.

So far in the beginner MPI tutorial, we have examined point-to-point communication, which is communication between two processes. This lesson is the start of the collective communication section. Collective communication is a method of communication which involves participation of all processes in a communicator. In this lesson, we will discuss the implications of collective communication… Read more »

Point-to-Point Communication Application – Random Walk

Posted by & filed under Beginner MPI.

It’s time to go through an application example using some of the concepts introduced in the sending and receiving tutorial and the MPI_Probe and MPI_Status lesson. The code for the application can be downloaded here or can be viewed/cloned on GitHub. The application simulates a process which I refer to as “random walking.” The basic… Read more »

Dynamic Receiving with MPI Probe (and MPI Status)

Posted by & filed under Beginner MPI.

In the previous lesson, I discussed how to use MPI_Send and MPI_Recv to perform standard point-to-point communication. I only covered how to send messages in which the length of the message was known beforehand. Although it is possible to send the length of the message as a separate send / recv operation, MPI natively supports… Read more »

MPI Send and Receive

Posted by & filed under Beginner MPI.

Sending and receiving are the two foundational concepts of MPI. Almost every single function in MPI can be implemented with basic send and receive calls. In this lesson, I will discuss how to use MPI’s blocking sending and receiving functions, and I will also overview other basic concepts associated with transmitting data using MPI. The… Read more »

MPI Hello World

Posted by & filed under Beginner MPI.

In this lesson, I will show you a basic MPI Hello World application and also discuss how to run an MPI program. The lesson will cover the basics of initializing MPI and running an MPI job across several processes. This lesson is intended to work with installations of MPICH2 (specifically 1.4). If you have not… Read more »

Installing MPICH2 on a Single Machine

Posted by & filed under Beginner MPI.

MPI is simply a standard which others follow in their implementation. Because of this, there are a wide variety of MPI implementations out there. One of the most popular implementations, MPICH2, will be used for all of the examples provided through this site. Users are free to use any implementation they wish, but only instructions… Read more »