We will post a series of resources to interesting tutorials and articles on parallel computing. In that series, the first is my favorite one by Blaise Barney of Lawrence Livermore National Laboratory.
From the author’s description, this tutorial covers the very basics of parallel computing, and is intended for someone who is just becoming acquainted with the subject. It begins with a brief overview, including concepts and terminology associated with parallel computing. The topics of parallel memory architectures and programming models are then explored. These topics are followed by a discussion on a number of issues related to designing parallel programs. The tutorial concludes with several examples of how to parallelize simple serial programs.
“In the simplest sense, parallel computing is the simultaneous use of multiple compute resources to solve a computational problem:
* To be run using multiple CPUs
* A problem is broken into discrete parts that can be solved concurrently
* Each part is further broken down to a series of instructions
* Instructions from each part execute simultaneously on different CPUs ”