By Stephen Toub
Partitioning is the act of splitting a data set to be processed in parallel by multiple logical processors. It is a fundamental aspect of parallel computing, as without partitioning everything would execute serially. There are, of course, a multitude of ways in which a data set may be partitioned, and which approach is best for a particular situation depends on a variety of potentially complicated factors.

The .NET Framework 4 invests heavily in helping developers to more easily write parallelized applications. As part of this, the Framework includes Parallel LINQ (PLINQ) for doing data-parallel processing with a familiar set of query operations, as well as the System.Threading.Tasks.Parallel class and its ForEach method, which enables the implementation of data-parallel loops. Both of these constructs need to be able to partition the provided data set, and both utilize multiple built-in algorithms and heuristics for partitioning data sets to be processed in parallel. However no set of built-in components could adequately represent the full spectrum of possible partitioning approaches. As such, both PLINQ and Parallel.ForEach support custom partitioning, enabling developers to implement their own partitioning algorithms, which PLINQ and Parallel.ForEach can then consume.

Full Story