Parallel Algorithms

This textbook is a concise introduction to the basic toolbox of structures that allow efficient organization and retrieval of data, key algorithms for problems on graphs, and generic techniques for modeling, understanding, and solving algorithmic problems. The authors aim for a balance between simplicity and efficiency, between theory and practice, and between classical results and the forefront of research. Individual chapters cover arrays and linked lists, hash tables and associative arrays, sorting and selection, priority queues, sorted sequences, graph representation, graph traversal, shortest paths, minimum spanning trees, optimization, collective communication and computation, and load balancing. The authors also discuss important issues such as algorithm engineering, memory hierarchies, algorithm libraries, and certifying algorithms. Moving beyond the sequential algorithms and data structures of the earlier related title, this book takes into account the paradigm shift towards the parallel processing required to solve modern performance-critical applications and how this impacts on the teaching of algorithms. The book is suitable for undergraduate and graduate students and professionals familiar with programming and basic mathematical language. Most chapters have the same basic structure: the authors discuss a problem as it occurs in a real-life situation, they illustrate the most important applications, and then they introduce simple solutions as informally as possible and as formally as necessary so the reader really understands the issues at hand. As they move to more advanced and optional issues, their approach gradually leads to a more mathematical treatment, including theorems and proofs. The book includes many examples, pictures, informal explanations, and exercises, and the implementation notes introduce clean, efficient implementations in languages such as C++ and Java.

Motivation It is now possible to build powerful single-processor and multiprocessor systems and use them efficiently for data processing, which has seen an explosive ex pansion in many areas of computer science and engineering. One approach to meeting the performance requirements of the applications has been to utilize the most powerful single-processor system that is available. When such a system does not provide the performance requirements, pipelined and parallel process ing structures can be employed. The concept of parallel processing is a depar ture from sequential processing. In sequential computation one processor is in volved and performs one operation at a time. On the other hand, in parallel computation several processors cooperate to solve a problem, which reduces computing time because several operations can be carried out simultaneously. Using several processors that work together on a given computation illustrates a new paradigm in computer problem solving which is completely different from sequential processing. From the practical point of view, this provides sufficient justification to investigate the concept of parallel processing and related issues, such as parallel algorithms. Parallel processing involves utilizing several factors, such as parallel architectures, parallel algorithms, parallel programming lan guages and performance analysis, which are strongly interrelated. In general, four steps are involved in performing a computational problem in parallel. The first step is to understand the nature of computations in the specific application domain.

Mathematics of Computing -- Parallelism.

Parallel algorithms Made Easy The complexity of today's applications coupled with the widespread use of parallel computing has made the design and analysis of parallel algorithms topics of growing interest. This volume fills a need in the field for an introductory treatment of parallel algorithms-appropriate even at the undergraduate level, where no other textbooks on the subject exist. It features a systematic approach to the latest design techniques, providing analysis and implementation details for each parallel algorithm described in the book. Introduction to Parallel Algorithms covers foundations of parallel computing; parallel algorithms for trees and graphs; parallel algorithms for sorting, searching, and merging; and numerical algorithms. This remarkable book: * Presents basic concepts in clear and simple terms * Incorporates numerous examples to enhance students' understanding * Shows how to develop parallel algorithms for all classical problems in computer science, mathematics, and engineering * Employs extensive illustrations of new design techniques * Discusses parallel algorithms in the context of PRAM model * Includes end-of-chapter exercises and detailed references on parallel computing. This book enables universities to offer parallel algorithm courses at the senior undergraduate level in computer science and engineering. It is also an invaluable text/reference for graduate students, scientists, and engineers in computer science, mathematics, and engineering.

Focusing on algorithms for distributed-memory parallel architectures, Parallel Algorithms presents a rigorous yet accessible treatment of theoretical models of parallel computation, parallel algorithm design for homogeneous and heterogeneous platforms, complexity and performance analysis, and essential notions of scheduling. The book extract

This contributed volume highlights two areas of fundamental interest in high-performance computing: core algorithms for important kernels and computationally demanding applications. The first few chapters explore algorithms, numerical techniques, and their parallel formulations for a variety of kernels that arise in applications. The rest of the volume focuses on state-of-the-art applications from diverse domains. By structuring the volume around these two areas, it presents a comprehensive view of the application landscape for high-performance computing, while also enabling readers to develop new applications using the kernels. Readers will learn how to choose the most suitable parallel algorithms for any given application, ensuring that theory and practicality are clearly connected. Applications using these techniques are illustrated in detail, including: Computational materials science and engineering Computational cardiovascular analysis Multiscale analysis of wind turbines and turbomachinery Weather forecasting Machine learning techniques Parallel Algorithms in Computational Science and Engineering will be an ideal reference for applied mathematicians, engineers, computer scientists, and other researchers who utilize high-performance computing in their work.

The Haifa 2000 Workshop on "Inherently Parallel Algorithms for Feasibility and Optimization and their Applications" brought together top scientists in this area. The objective of the Workshop was to discuss, analyze and compare the latest developments in this fast growing field of applied mathematics and to identify topics of research which are of special interest for industrial applications and for further theoretical study. Inherently parallel algorithms, that is, computational methods which are, by their mathematical nature, parallel, have been studied in various contexts for more than fifty years. However, it was only during the last decade that they have mostly proved their practical usefulness because new generations of computers made their implementation possible in order to solve complex feasibility and optimization problems involving huge amounts of data via parallel processing. These led to an accumulation of computational experience and theoretical information and opened new and challenging questions concerning the behavior of inherently parallel algorithms for feasibility and optimization, their convergence in new environments and in circumstances in which they were not considered before their stability and reliability. Several research groups all over the world focused on these questions and it was the general feeling among scientists involved in this effort that the time has come to survey the latest progress and convey a perspective for further development and concerted scientific investigations. Thus, the editors of this volume, with the support of the Israeli Academy for Sciences and Humanities, took the initiative of organizing a Workshop intended to bring together the leading scientists in the field. The current volume is the Proceedings of the Workshop representing the discussions, debates and communications that took place. Having all that information collected in a single book will provide mathematicians and engineers interested in the theoretical and practical aspects of the inherently parallel algorithms for feasibility and optimization with a tool for determining when, where and which algorithms in this class are fit for solving specific problems, how reliable they are, how they behave and how efficient they were in previous applications. Such a tool will allow software creators to choose ways of better implementing these methods by learning from existing experience.

There is a software gap between the hardware potential and the performance that can be attained using today's software parallel program development tools. The tools need manual intervention by the programmer to parallelize the code. Programming a parallel computer requires closely studying the target algorithm or application, more so than in the traditional sequential programming we have all learned. The programmer must be aware of the communication and data dependencies of the algorithm or application. This book provides the techniques to explore the possible ways to program a parallel computer for a given application.