Compiling Parallel Loops For High Performance Computers

4. 2 Code Segments . . . . . . . . . . . . . . . 96 4. 3 Determining Communication Parameters . 99 4. 4 Multicast Communication Overhead · 103 4. 5 Partitioning . . . . . . · 103 4. 6 Experimental Results . 117 4. 7 Conclusion. . . . . . . · 121 5 COLLECTIVE PARTITIONING AND REMAPPING FOR MULTIPLE LOOP NESTS 125 5. 1 Introduction. . . . . . . . . 125 5. 2 Program Enclosure Trees. . 128 5. 3 The CPR Algorithm . . 132 5. 4 Experimental Results. . 141 5. 5 Conclusion. . 146 BIBLIOGRAPHY. 149 INDEX . . . . . . . . 157 LIST OF FIGURES Figure 1. 1 The Butterfly Architecture. . . . . . . . . . 5 1. 2 Example of an iterative data-parallel loop . . 7 1. 3 Contiguous tiling and assignment of an iteration space. 13 2. 1 Communication along a line segment. . . 24 2. 2 Access pattern for the access offset, (3,2). 25 2. 3 Decomposing an access vector along an orthogonal basis set of vectors. . . . . . . . . . . . . . . . . . . 26 2. 4 An analysis of communication patterns. 29 2. 5 Decomposing a vector along two separate basis sets of vectors. 31 2. 6 Cache lines aligning with borders. 33 2. 7 Cache lines not aligned with borders. 34 2. 8 nh is the difference of nd and nb. 42 2. 9 nh is the sum of nd and nb. 42 2. 10 The ADAPT system. 44 2. 11 Code segment used in experiments. . 46 2. 12 Execution rates for various partitions. 47 2. 13 Execution time of partitions on Multimax. 48 2. 14 Performance increase as processing power increases. 49 2. 15 Percentage miss ratios for various aspect ratios and line sizes.

Software -- Operating Systems.

This book constitutes the thoroughly refereed post-proceedings of the 15th International Workshop on Languages and Compilers for Parallel Processing, LCPC 2002, held in College Park, MD, USA in July 2002. The 26 revised full papers presented were carefully selected during two rounds of reviewing and improvement from 32 submissions. All current issues in parallel processing are addressed, in particular memory-constrained computation, compiler optimization, performance studies, high-level languages, programming language consistency models, dynamic parallelization, parallelization of data mining algorithms, parallelizing compilers, garbage collection algorithms, and evaluation of iterative compilation.

The second half of the 1970s was marked with impressive advances in array/vector architectures and vectorization techniques and compilers. This progress continued with a particular focus on vector machines until the middle of the 1980s. The major ity of supercomputers during this period were register-to-register (Cray 1) or memory-to-memory (CDC Cyber 205) vector (pipelined) machines. However, the increasing demand for higher computational rates lead naturally to parallel comput ers and software. Through the replication of autonomous processors in a coordinated system, one can skip over performance barriers due technology limitations. In princi ple, parallelism offers unlimited performance potential. Nevertheless, it is very difficult to realize this performance potential in practice. So far, we have seen only the tip of the iceberg called "parallel machines and parallel programming". Parallel programming in particular is a rapidly evolving art and, at present, highly empirical. In this book we discuss several aspects of parallel programming and parallelizing compilers. Instead of trying to develop parallel programming methodologies and paradigms, we often focus on more advanced topics assuming that the reader has an adequate background in parallel processing. The book is organized in three main parts. In the first part (Chapters 1 and 2) we set the stage and focus on program transformations and parallelizing compilers. The second part of this book (Chapters 3 and 4) discusses scheduling for parallel machines from the practical point of view macro and microtasking and supporting environments). Finally, the last part (Le.

The state of the art of high-performance computing Prominent researchers from around the world have gathered to present the state-of-the-art techniques and innovations in high-performance computing (HPC), including: * Programming models for parallel computing: graph-oriented programming (GOP), OpenMP, the stages and transformation (SAT) approach, the bulk-synchronous parallel (BSP) model, Message Passing Interface (MPI), and Cilk * Architectural and system support, featuring the code tiling compiler technique, the MigThread application-level migration and checkpointing package, the new prefetching scheme of atomicity, a new "receiver makes right" data conversion method, and lessons learned from applying reconfigurable computing to HPC * Scheduling and resource management issues with heterogeneous systems, bus saturation effects on SMPs, genetic algorithms for distributed computing, and novel task-scheduling algorithms * Clusters and grid computing: design requirements, grid middleware, distributed virtual machines, data grid services and performance-boosting techniques, security issues, and open issues * Peer-to-peer computing (P2P) including the proposed search mechanism of hybrid periodical flooding (HPF) and routing protocols for improved routing performance * Wireless and mobile computing, featuring discussions of implementing the Gateway Location Register (GLR) concept in 3G cellular networks, maximizing network longevity, and comparisons of QoS-aware scatternet scheduling algorithms * High-performance applications including partitioners, running Bag-of-Tasks applications on grids, using low-cost clusters to meet high-demand applications, and advanced convergent architectures and protocols High-Performance Computing: Paradigm and Infrastructure is an invaluable compendium for engineers, IT professionals, and researchers and students of computer science and applied mathematics.

Language, Compilers and Run-time Systems for Scalable Computers contains 20 articles based on presentations given at the third workshop of the same title, and 13 extended abstracts from the poster session. Starting with new developments in classical problems of parallel compiler design, such as dependence analysis and an exploration of loop parallelism, the book goes on to address the issues of compiler strategy for specific architectures and programming environments. Several chapters investigate support for multi-threading, object orientation, irregular computation, locality enhancement, and communication optimization. Issues of the interface between language and operating system support are also discussed. Finally, the load balance issues are discussed in different contexts, including sparse matrix computation and iteratively balanced adaptive solvers for partial differential equations. Some additional topics are also discussed in the extended abstracts. Each chapter provides a bibliography of relevant papers and the book can thus be used as a reference to the most up-to-date research in parallel software engineering.

In 2008 the Workshop on Languages and Compilers for Parallel Computing left the USA to celebrate its 21st anninversary in Edmonton, Alberta, Canada. Following its long-established tradition, the workshop focused on topics at the frontierofresearchanddevelopmentinlanguages,optimizingcompilers,appli- tions, and programming models for high-performance computing. While LCPC continues to focus on parallel computing, the 2008 edition included the pres- tation of papers on program analysis that are precursors of high performance in parallel environments. LCPC 2008 received 35 paper submissions. Eachpaper received at least three independent reviews, and then the papers and the referee comments were d- cussed during a Program Committee meeting. The PC decided to accept 18 papers as regular papers and 6 papers as short papers. The short papers appear at the end of this volume. The LCPC 2008 program was fortunate to include two keynote talks. Keshav Pingali’s talk titled “Amorphous Data Parallelism in Irregular Programs” - gued that irregular programs have data parallelism in the iterative processing of worklists. Pingali described the Galois system developed at The University of Texas at Austin to exploit this kind of amorphous data parallelism. The second keynote talk, “Generic ParallelAlgorithms in Threading Building Bocks (TBB),” presented by Arch Robison from Intel Corporation addressed very practical aspects of using TBB, a production C++ library, for generic p- allel programming and contrasted TBB with the Standard Template Library (STL).

This book constitutes the thoroughly refereed post-proceedings of the 14th International Workshop on Languages and Compilers for Parallel Computing, LCPC 2001, held in Lexington, KY, USA, in August 1-3, 2001. The 28 revised full papers presented were carefully selected during two rounds of reviewing and improvement. All current issues in parallel processing are addressed, in particular compiler optimization, HP Java programming, power-aware parallel architectures, high performance applications, power management of mobile computers, data distribution, shared memory systems, load balancing, garbage collection, parallel components, job scheduling, dynamic parallelization, cache optimization, specification, and dataflow analysis.