We provided many resources of parallel programming tutorials. The following are the ones we have linked so far.
Basic parallel computing tutorial
MapReduce tutorial
Cell processor programming
OpenMP tutorial
PThreads Tutorials
Intel Threading Building Blocks
In Part 7, we visit Message Passing Interface (MPI), the de facto standard for writing parallel programs running on a distributed memory system, such as a compute cluster, and is widely implemented.
Our choice for basic introductory MPI tutorial is again from Blaise Barney’s collection of tutorials. An excerpt from this tutorial:
“The Message Passing Interface Standard (MPI) is a message passing library standard based on the consensus of the MPI Forum, which has over 40 participating organizations, including vendors, researchers, software library developers, and users. The goal of the Message Passing Interface is to establish a portable, efficient, and flexible standard for message passing that will be widely used for writing message passing programs. As such, MPI is the first standardized, vendor independent, message passing library. The advantages of developing message passing software using MPI closely match the design goals of portability, efficiency, and flexibility.”
MCS Divison of Argonne National Lab provides a page with many links to MPI tutorials here. Some of these tutorials have example code available for downloading.
Some More Interesting Sources
Tutorial on MPI: The Message-Passing Interface by William Gropp
Getting started with LAM/MPI
NERSC MPI Tutorial
MPI Exercises
Basic MPI Tutorial
Adaptive Message Passing Interface (AMPI) Tutorial [ppt slides] [AMPI]
Useful Links
MPICH2 Website
Open MPI Website
Fault Tolerant MPI
LAM/MPI
Message Passing Interface Forum
MPI 3.0 Standardization Effort
MPI Specification Documents
The Evolution of MPI by Bill Gropp
Microsoft MPI
Books
* Gropp, William; Lusk, Ewing; Skjellum, Anthony. (1999) Using MPI, 2nd Edition: portable Parallel Programming with the Message Passing Interface. MIT Press In Scientific And Engineering Computation Series, Cambridge, MA, USA. 395 pp. ISBN 978-0-262-57132-6
* Gropp, William; R Thakur, E Lusk (1999) Using MPI-2: Advanced Features of the Message Passing Interface - MIT Press Cambridge, MA, USA ISBN 0-262-57133-1
* Pacheco, Peter S. (1997) Parallel Programming with MPI.[3] 500 pp. Morgan Kaufmann ISBN 1558603395
5 responses so far ↓
1 Twitted by igorsukharev // Jul 4, 2009 at 8:48 am
[...] This post was Twitted by igorsukharev [...]
2 Parallel Programming Tutorial Series - Part 8 - CUDA // Aug 1, 2009 at 7:25 am
[...] We provided many resources of parallel programming tutorials. The following are the ones we have linked so far. Basic parallel computing tutorial MapReduce tutorial Cell processor programming OpenMP tutorial PThreads Tutorials Intel Threading Building Blocks MPI [...]
3 Parallel Programming Tutorial Series - Part 9 - OpenC4 // Aug 24, 2009 at 5:37 am
[...] Cell processor programming OpenMP tutorial PThreads Tutorials Intel Threading Building Blocks MPI [...]
4 Multicore Review: Best Multicore Posts of 2009 // Dec 20, 2009 at 8:33 am
[...] into Basic Parallel computing, MapReduce, Cell Processor Programming, OpenMP, pthreads, Intel TBB, MPI, CUDA, and OpenCL. Many readers found these very useful, especially CUDA and OpenCL ones in the [...]
5 Parallel Programming Tutorial Series - Part 9 - OpenCL (Updated) // Mar 26, 2011 at 6:18 am
[...] Cell processor programming OpenMP tutorial PThreads Tutorials Intel Threading Building Blocks MPI [...]