Introduction To Parallel Computing

In modern computer science, there exists no truly sequential computing system; and most advanced programming is parallel programming. This is particularly evident in modern application domains like scientific computation, data science, machine intelligence, etc. This lucid introductory textbook will be invaluable to students of computer science and technology, acting as a self-contained primer to parallel programming. It takes the reader from introduction to expertise, addressing a broad gamut of issues. It covers different parallel programming styles, describes parallel architecture, includes parallel programming frameworks and techniques, presents algorithmic and analysis techniques and discusses parallel design and performance issues. With its broad coverage, the book can be useful in a wide range of courses; and can also prove useful as a ready reckoner for professionals in the field.

A complete source of information on almost all aspects of parallel computing from introduction, to architectures, to programming paradigms, to algorithms, to programming standards. It covers traditional Computer Science algorithms, scientific computing algorithms and data intensive algorithms.

Advancements in microprocessor architecture, interconnection technology, and software development have fueled rapid growth in parallel and distributed computing. However, this development is only of practical benefit if it is accompanied by progress in the design, analysis and programming of parallel algorithms. This concise textbook provides, in one place, three mainstream parallelization approaches, Open MPP, MPI and OpenCL, for multicore computers, interconnected computers and graphical processing units. An overview of practical parallel computing and principles will enable the reader to design efficient parallel programs for solving various computational problems on state-of-the-art personal computers and computing clusters. Topics covered range from parallel algorithms, programming tools, OpenMP, MPI and OpenCL, followed by experimental measurements of parallel programs’ run-times, and by engineering analysis of obtained results for improved parallel execution performances. Many examples and exercises support the exposition.

A comprehensive guide for students and practitioners to parallel computing models, processes, metrics, and implementation in MPI and OpenMP.

This book introduces students to the full gamut of different parallel programming styles and their relationship to hardware architecture.

"Embedded Computing is enthralling in its clarity and exhilarating in its scope. If the technology you are working on is associated with VLIWs or "embedded computing", then clearly it is imperative that you read this book. If you are involved in computer system design or programming, you must still read this book, because it will take you to places where the views are spectacular. You don't necessarily have to agree with every point the authors make, but you will understand what they are trying to say, and they will make you think.” From the Foreword by Robert Colwell, R&E Colwell & Assoc. Inc The fact that there are more embedded computers than general-purpose computers and that we are impacted by hundreds of them every day is no longer news. What is news is that their increasing performance requirements, complexity and capabilities demand a new approach to their design. Fisher, Faraboschi, and Young describe a new age of embedded computing design, in which the processor is central, making the approach radically distinct from contemporary practices of embedded systems design. They demonstrate why it is essential to take a computing-centric and system-design approach to the traditional elements of nonprogrammable components, peripherals, interconnects and buses. These elements must be unified in a system design with high-performance processor architectures, microarchitectures and compilers, and with the compilation tools, debuggers and simulators needed for application development. In this landmark text, the authors apply their expertise in highly interdisciplinary hardware/software development and VLIW processors to illustrate this change in embedded computing. VLIW architectures have long been a popular choice in embedded systems design, and while VLIW is a running theme throughout the book, embedded computing is the core topic. Embedded Computing examines both in a book filled with fact and opinion based on the authors many years of R&D experience. Features: · Complemented by a unique, professional-quality embedded tool-chain on the authors' website, http://www.vliw.org/book · Combines technical depth with real-world experience · Comprehensively explains the differences between general purpose computing systems and embedded systems at the hardware, software, tools and operating system levels. · Uses concrete examples to explain and motivate the trade-offs.

THE CONTEXT OF PARALLEL PROCESSING The field of digital computer architecture has grown explosively in the past two decades. Through a steady stream of experimental research, tool-building efforts, and theoretical studies, the design of an instruction-set architecture, once considered an art, has been transformed into one of the most quantitative branches of computer technology. At the same time, better understanding of various forms of concurrency, from standard pipelining to massive parallelism, and invention of architectural structures to support a reasonably efficient and user-friendly programming model for such systems, has allowed hardware performance to continue its exponential growth. This trend is expected to continue in the near future. This explosive growth, linked with the expectation that performance will continue its exponential rise with each new generation of hardware and that (in stark contrast to software) computer hardware will function correctly as soon as it comes off the assembly line, has its down side. It has led to unprecedented hardware complexity and almost intolerable dev- opment costs. The challenge facing current and future computer designers is to institute simplicity where we now have complexity; to use fundamental theories being developed in this area to gain performance and ease-of-use benefits from simpler circuits; to understand the interplay between technological capabilities and limitations, on the one hand, and design decisions based on user and application requirements on the other.

In the last few years, courses on parallel computation have been developed and offered in many institutions in the UK, Europe and US as a recognition of the growing significance of this topic in mathematics and computer science. There is a clear need for texts that meet the needs of students and lecturers and this book, based on the author's lecture at ETH Zurich, is an ideal practical student guide to scientific computing on parallel computers working up from a hardware instruction level, to shared memory machines, and finally to distributed memory machines. Aimed at advanced undergraduate and graduate students in applied mathematics, computer science, and engineering, subjects covered include linear algebra, fast Fourier transform, and Monte-Carlo simulations, including examples in C and, in some cases, Fortran. This book is also ideal for practitioners and programmers.