In this second part of a series of articles written by Tim Mattson about the jargon that has sprung up around parallel computing, SMP, SIMD, MPP, NUMA, and MIMD, are discussed.
* Single instruction stream operating on multiple streams of data or SIMD
* Multiple streams of instructions operating on multiple streams of data or MIMD
For a SIMD computer, the parallelism is in the data. You have one program counter that moves through a stream of instructions that operates on multiple elements of data.
For MIMD computers, each processing element has its own program counter and hence runs an independent stream of instructions.
1. Symmetric Multiprocessor or SMP computers: the cost of accessing an address in memory is the same for each processor. Furthermore, the processors are all equal in the eyes of the operation system.
2. Non-uniform Memory Architecture or NUMA computers: the cost of accessing a given address in memory varies from one processor to another.
