Message Passing Interface (MPI)

By: Aryan Yadav

Introduction:

A typical specification of a message-passing interface for parallel processing in distributed memory systems is called the Message Passing Interface (MPI). A programming language isn’t really MPI. It is a library of functions that programmers can use to create parallel applications in C, C++, or Fortran. An MPI communicator can be temporarily established with MPI, allowing numerous processes to operate simultaneously on various cluster nodes. Each process executes independently from the other processes, has a distinct MPI rank to identify it, and has its own memory space. To share data, processes talk to one another by exchanging messages. When a program task is divided into manageable bits and distributed across the processes, each process completes its portion in parallel. The message-passing parallel programming model is the main focus of MPI; it entails cooperative actions on each process to transfer data from one process’ address space to another’s.

MPI Communication Methods:

1. Point-to-Point Communications: In MPI, point-to-point communication is the most popular type of communication. It entails passing a message from one process to a specific process inside the same communicator. Point-to-Point communication using MPI is available in both blocking (synchronous) and non-blocking (asynchronous) modes. When communicating in a blocking manner, an MPI process delivers a message to another MPI process and then waits for the receiving process to fully and accurately receive the message before continuing. A transmitting process using non-blocking communication, on the other hand, sends a message to another MPI process and continues working without waiting to make sure the message was correctly received by the receiving process.

Point-to-Point Communications

2. Collective Communication: A process that broadcasts a message to all programs in the same communicator, including itself, while using this sort of MPI communication mechanism.

Collective Communication

3. One-sided communication: With the MPI One-sided communication mechanism, a process can interact with another process directly and access its memory.

General MPI Program Structure:

Simple Code Examples in C/Fortan:

MPI Eg. in C++

MPI Eg. in Fortan

Reasons why we are using MPI:

1. Standardization: The first and only message-passing library that counts as a standard is MPI. It works with almost all HPC platforms. It has essentially taken the place of all earlier message-passing libraries.
2. Performance Possibilities: In order to maximize performance, vendor implementations should be able to take advantage of built-in hardware capabilities. It is free for any implementation to create optimal algorithms.
3. Functionality: MPI-3 defines around 430 procedures, most of which are also found in MPI-2 and MPI-1.
4. Portability: When moving an application to a new platform that supports (and complies with) the MPI standard, there is little to no need to change the source code.
5. Availability: Both vendor-provided and publicly accessible implementations come in a wide range.

History & Evolution:

The MPI Forum released the MPI “standard” in May 1994, and it was enhanced in June 1995. The University of Tennessee issued “MPI: A Message-Passing Standard,” which can be found on the MPI Forum, and it serves as the document’s definition. If you are unfamiliar with MPI, you should definitely consult this article for information on its syntax since this workshop will only briefly touch on it to explain a few use scenarios.

In 1997, MPI-2.0 was finished, while MPI-3.0 was accomplished in 2012. 2015 saw the completion of MPI-3.1, which offers minor improvements and fixes to MPI-3.0. Given the significant number of parallel applications that rely on MPI and the lengthy history of a stable interface for MPI, it is likely that the core API will remain unaltered for many more years.

The Intel MPI libraries installed on Stampede2 and Frontera implement MPI 3, which is still the most extensively used standard. We can continue to anticipate additional extensions embracing new architectures and parallel programming paradigms, as was the case for MPI-2 and MPI-3, given the dynamic nature of HPC design.

The release day for MPI-4.0 was June 9, 2021. The primary focus is on MPI 4.1 and MPI 5.0.