In this part of a series of articles by Tim Mattson on introducing the parallel computing jargon, parallel computing software, and techniques of writing parallel software.

“The software world has been quite busy with parallel computing. On the MPP front, we developed in the mid 90’s, a number of message passing APIs eventually converging on a single standard (MPI). For shared-memory, multiprocessor computing, a similar process unfolded with convergence around two standards;

* pthreads for explicit control over threads and
* OpenMP for directive driven multithreading.

And while these standards-based approaches were becoming entrenched, every imaginable flavor of parallel programming was tried: data parallel (HPF), functional (SISAL), concurrent logic programming (Parlog), object oriented programming (POOMA), and many others. In each case, the high level goal was the same … to make parallel programming easier and less error prone.

Regardless of the parallel language, the basic idea behind parallel software is the same. We can think of it in terms of a consistent four-step program:

1. Find the concurrency in your problem.
2. Expose the concurrency, i.e. restructure code to make potentially concurrent code truly concurrent.
3. Exploit the concurrency by adding parallel constructs to your program using a parallel programming notation.
4. Execute the program in parallel and tune.

Full Story