by Michael Wolfe, Compiler Engineer, The Portland Group, Inc.
As we move towards exascale, we want to take advantage of all of these. We need to expose parallelism at all the levels, either explicitly in the program, or implicitly within the compiler. We need to express this parallelism, in a language and in the generated code. And we need to exploit the parallelism, efficiently and effectively, at runtime on the target machine. Performance is important; in fact, performance is key. The only reason we are looking at parallelism is for higher performance. This is a point I have made in the past, and I’ll say it again: the only reason to program in parallel is for higher performance. (Someone at SC10 responded that another reason to use parallelism is for redundancy, for error detection. I should have replied that redundancy is good, but you do the redundant computations in parallel to get the benefits of redundant computation without performance degradation.)

Exposing Parallelism
Someone has to find or create the parallelism in the program. At tera and petascale, we’ve mostly focused on data parallelism: large datasets, where the program can operate on separate data in parallel. At exascale, we’re likely to want even more parallelism, such as running coupled models in parallel. We’ll do the structural mechanics model concurrently with the heat transfer and deformation physics models. Since these models are coupled, it’s much easier to separate the problems and iterate through them one at a time; it will be a challenge to structure the code so they can run in parallel with each other. Parallelism at this high a level has to be exposed in the application design.

Full Story