Distributed Algorithms

A comprehensive guide to distributed algorithms that emphasizes examples and exercises rather than mathematical argumentation.

A comprehensive guide to distributed algorithms that emphasizes examples and exercises rather than mathematical argumentation. This book offers students and researchers a guide to distributed algorithms that emphasizes examples and exercises rather than the intricacies of mathematical models. It avoids mathematical argumentation, often a stumbling block for students, teaching algorithmic thought rather than proofs and logic. This approach allows the student to learn a large number of algorithms within a relatively short span of time. Algorithms are explained through brief, informal descriptions, illuminating examples, and practical exercises. The examples and exercises allow readers to understand algorithms intuitively and from different perspectives. Proof sketches, arguing the correctness of an algorithm or explaining the idea behind fundamental results, are also included. An appendix offers pseudocode descriptions of many algorithms. Distributed algorithms are performed by a collection of computers that send messages to each other or by multiple software threads that use the same shared memory. The algorithms presented in the book are for the most part “classics,” selected because they shed light on the algorithmic design of distributed systems or on key issues in distributed computing and concurrent programming. Distributed Algorithms can be used in courses for upper-level undergraduates or graduate students in computer science, or as a reference for researchers in the field.

An Introduction to Distributed Algorithms takes up some of the main concepts and algorithms, ranging from basic to advanced techniques and applications, that underlie the programming of distributed-memory systems such as computer networks, networks of work-stations, and multiprocessors. Written from the broad perspective of distributed-memory systems in general it includes topics such as algorithms for maximum flow, programme debugging, and simulation that do not appear in more orthodox texts on distributed algorithms.

Introduction : distributed systems - The model - Communication protocols - Routing algorithms - Deadlock-free packet switching - Wave and traversal algorithms - Election algorithms - Termination detection - Anonymous networks - Snapshots - Sense of direction and orientation - Synchrony in networks - Fault tolerance in distributed systems - Fault tolerance in asynchronous systems - Fault tolerance in synchronous systems - Failure detection - Stabilization.

AN ELABORATE YET BEGINNER-FRIENDLY GUIDE TO DISTRIBUTED ALGORITHMS Distributed Algorithms, a non-trivial and highly evolving field of active research, is often presented in most publications using a heavy accompaniment of mathematical techniques and notations. Aimed squarely at beginners as well as experienced practitioners, this book attempts to demystify and explicate the subject of distributed algorithms using a highly expansive and verbose style of treatment. Covering scores of landmark algorithms in the field of distributed computing, the approach is to present and analyse each topic using a minimum of mathematical exposition, reverting instead to a fluid style of description in plain English. A mathematical presentation is avoided altogether whenever such a move does not reduce the quality of the analysis at hand. Elsewhere, the effort always is to talk and guide the reader through the relevant math without resorting to a series of equations. To backup such a style of treatment, each topic is accompanied by a multitude of examples, flowcharts, and diagrams. The book is divided into three parts; the first part deals with fundamentals, the second and largest of the three is all about algorithms specific to message passing networks, while the last one focuses on shared memory algorithms. The beginning of the book dedicates a few chapters to the basics - including a quick orientation on the underlying platform, i.e. distributed systems, their characteristics, advantages, challenges, and so on. Some of the earlier chapters also address basic algorithms and techniques relevant to distributed computing environments before moving on to progressively complex algorithms and results - en route to the later chapters in the second part which deal with widely used 'industrial-strength' protocols such as Paxos and Raft. The third part of the book does assume a basic orientation towards computer programming, and presents numerous shared memory algorithms where each one is accompanied by a detailed description, analysis, pseudo code, and in some cases, code (C or C++). Whenever actual code is used, the syntax is kept as basic as possible - incorporating only elementary features of the language - so that newbie programmers can follow the presentation smoothly. Lastly, the target audience of the book is wide enough to cover beginners such as students or graduates joining the industry, experienced professionals wishing to migrate from monolithic frameworks to distributed ones, as well as readers with years of experience on the subject of distributed computing. The style of presentation is selected with the first two classes of readers in mind: those who wish to quickly ramp up on the subject of distributed algorithms for professional reasons or personal ones. While staying true to the stated aim, the book does not shy away from dealing with complex topics. A concise list of content information follows: Introduction to distributed systems Properties of distributed data stores and Brewer's theorem Building blocks: unicast, broadcast, algorithms in cubes Leader election algorithms: for ring/generic networks Consensus algorithms: synchronous/asynchronous variants for message passing and shared memory systems Distributed commits, Paxos, Raft Graph algorithms Routing algorithms Time and order Mutual exclusion: for message passing networks Debug algorithms: snapshot, deadlock/termination detection Shared memory: practical problems, mutual exclusion, consensus, resource allocation About the author Fourré Sigs is an industry veteran with over 25 years of experience in systems programming, networking, and highly scalable and secure distributed service architectures.

Distributed computing is at the heart of many applications. It arises as soon as one has to solve a problem in terms of entities -- such as processes, peers, processors, nodes, or agents -- that individually have only a partial knowledge of the many input parameters associated with the problem. In particular each entity cooperating towards the common goal cannot have an instantaneous knowledge of the current state of the other entities. Whereas parallel computing is mainly concerned with 'efficiency', and real-time computing is mainly concerned with 'on-time computing', distributed computing is mainly concerned with 'mastering uncertainty' created by issues such as the multiplicity of control flows, asynchronous communication, unstable behaviors, mobility, and dynamicity. While some distributed algorithms consist of a few lines only, their behavior can be difficult to understand and their properties hard to state and prove. The aim of this book is to present in a comprehensive way the basic notions, concepts, and algorithms of distributed computing when the distributed entities cooperate by sending and receiving messages on top of an asynchronous network. The book is composed of seventeen chapters structured into six parts: distributed graph algorithms, in particular what makes them different from sequential or parallel algorithms; logical time and global states, the core of the book; mutual exclusion and resource allocation; high-level communication abstractions; distributed detection of properties; and distributed shared memory. The author establishes clear objectives per chapter and the content is supported throughout with illustrative examples, summaries, exercises, and annotated bibliographies. This book constitutes an introduction to distributed computing and is suitable for advanced undergraduate students or graduate students in computer science and computer engineering, graduate students in mathematics interested in distributed computing, and practitioners and engineers involved in the design and implementation of distributed applications. The reader should have a basic knowledge of algorithms and operating systems.

This text is based on a simple and fully reactive computational model that allows for intuitive comprehension and logical designs. The principles and techniques presented can be applied to any distributed computing environment (e.g., distributed systems, communication networks, data networks, grid networks, internet, etc.). The text provides a wealth of unique material for learning how to design algorithms and protocols perform tasks efficiently in a distributed computing environment.

Distributed Computing is rapidly becoming the principal computing paradigm in diverse areas of computing, communication, and control. Processor clusters, local and wide area networks, and the information highway evolved a new kind of problems which can be solved with distributed algorithms. In this textbook a variety of distributed algorithms are presented independently of particular programming languages or hardware, using the graphically suggestive technique of Petri nets which is both easy to comprehend intuitively and formally rigorous. By means of temporal logic the author provides surprisingly simple yet powerful correctness proofs for the algorithms. The scope of the book ranges from distributed control and synchronization of two sites up to algorithms on any kind of networks. Numerous examples show that description and analysis of distributed algorithms in this framework are intuitive and technically transparent.