High Performance Computing Demystified

High Performance Computing Demystified provides an overview of high performance resources and their applications across many disciplines. This book is organized into five parts encompassing 16 chapters that cover the principles, mode of operation, and practical aspects of supercomputers. The first and second parts provide a brief history of high performance computing and describe the “basic parts needed to build high performance computers, including high performance microprocessors and network topologies. The third part examines the features of multiprocessor architectures of high performance, such as the large number crunchers, massively parallel processing machines, and networks of workstations. The fourth part deals with the software paradigms for high performance, while the fifth part looks into the high performance computing resources that are available to the public, with some guide to accessing those resources. This book is intended primarily for engineers and business managers who have a basic understanding of computers and would like to learn about high performance computing.

High Performance Computing: Modern Systems and Practices is a fully comprehensive and easily accessible treatment of high performance computing, covering fundamental concepts and essential knowledge while also providing key skills training. With this book, domain scientists will learn how to use supercomputers as a key tool in their quest for new knowledge. In addition, practicing engineers will discover how supercomputers can employ HPC systems and methods to the design and simulation of innovative products, and students will begin their careers with an understanding of possible directions for future research and development in HPC. Those who maintain and administer commodity clusters will find this textbook provides essential coverage of not only what HPC systems do, but how they are used. Covers enabling technologies, system architectures and operating systems, parallel programming languages and algorithms, scientific visualization, correctness and performance debugging tools and methods, GPU accelerators and big data problems Provides numerous examples that explore the basics of supercomputing, while also providing practical training in the real use of high-end computers Helps users with informative and practical examples that build knowledge and skills through incremental steps Features sidebars of background and context to present a live history and culture of this unique field Includes online resources, such as recorded lectures from the authors’ HPC courses

High Performance Computing Systems and Applications contains fully refereed papers from the 15th Annual Symposium on High Performance Computing. These papers cover both fundamental and applied topics in HPC: parallel algorithms, distributed systems and architectures, distributed memory and performance, high level applications, tools and solvers, numerical methods and simulation, advanced computing systems, and the emerging area of computational grids. High Performance Computing Systems and Applications is suitable as a secondary text for graduate level courses, and as a reference for researchers and practitioners in industry.

Complex calculations, like training deep learning models or running large-scale simulations, can take an extremely long time. Efficient parallel programming can save hours--or even days--of computing time. Parallel and High Performance Computing shows you how to deliver faster run-times, greater scalability, and increased energy efficiency to your programs by mastering parallel techniques for multicore processor and GPU hardware. about the technology Modern computing hardware comes equipped with multicore CPUs and GPUs that can process numerous instruction sets simultaneously. Parallel computing takes advantage of this now-standard computer architecture to execute multiple operations at the same time, offering the potential for applications that run faster, are more energy efficient, and can be scaled to tackle problems that demand large computational capabilities. But to get these benefits, you must change the way you design and write software. Taking advantage of the tools, algorithms, and design patterns created specifically for parallel processing is essential to creating top performing applications. about the book Parallel and High Performance Computing is an irreplaceable guide for anyone who needs to maximize application performance and reduce execution time. Parallel computing experts Robert Robey and Yuliana Zamora take a fundamental approach to parallel programming, providing novice practitioners the skills needed to tackle any high-performance computing project with modern CPU and GPU hardware. Get under the hood of parallel computing architecture and learn to evaluate hardware performance, scale up your resources to tackle larger problem sizes, and deliver a level of energy efficiency that makes high performance possible on hand-held devices. When you''re done, you''ll be able to build parallel programs that are reliable, robust, and require minimal code maintenance. This book is unique in its breadth, with discussions of parallel algorithms, techniques to successfully develop parallel programs, and wide coverage of the most effective languages for the CPU and GPU. The programming paradigms include MPI, OpenMP threading, and vectorization for the CPU. For the GPU, the book covers OpenMP and OpenACC directive-based approaches and the native-based CUDA and OpenCL languages. what''s inside Steps for planning a new parallel project Choosing the right data structures and algorithms Addressing underperforming kernels and loops The differences in CPU and GPU architecture about the reader For experienced programmers with proficiency in a high performance computing language such as C, C++, or Fortran. about the authors Robert Robey has been active in the field of parallel computing for over 30 years. He works at Los Alamos National Laboratory, and has previously worked at the University of New Mexico, where he started up the Albuquerque High Performance Computing Center. Yuliana Zamora has lectured on efficient programming of modern hardware at national conferences, based on her work developing applications running on tens of thousands of processing cores and the latest GPU architectures.

Because hardware technology appears unable to provide continuing speed increases, and because parallel architectures and software are not sufficiently developed to provide the practical solutions that have seemed tantalizingly close for some time, computing as a whole is at a crossroads. Even after more than a decade of commercial development, no standard or widely accepted systems have emerged. However, this text defines practical parallelism tests and suggests how they can be passed by giving specific technical suggestions and outlining policy steps that should be taken. Also suggested are methods for evolving better systems from those already in use and applying the definitions of what is needed as rules of selection.

The book discusses the fundamentals of high-performance computing. The authors combine visualization, comprehensibility, and strictness in their material presentation, and thus influence the reader towards practical application and learning how to solve real computing problems. They address both key approaches to programming modern computing systems: multithreading-based parallelizing in shared memory systems, and applying message-passing technologies in distributed systems. The book is suitable for undergraduate and graduate students, and for researchers and practitioners engaged with high-performance computing systems. Each chapter begins with a theoretical part, where the relevant terminology is introduced along with the basic theoretical results and methods of parallel programming, and concludes with a list of test questions and problems of varying difficulty. The authors include many solutions and hints, and often sample code.

High Performance Computing: Programming and Applications presents techniques that address new performance issues in the programming of high performance computing (HPC) applications. Omitting tedious details, the book discusses hardware architecture concepts and programming techniques that are the most pertinent to application developers for achievi