Randomness And Complexity

Computability and complexity theory are two central areas of research in theoretical computer science. This book provides a systematic, technical development of "algorithmic randomness" and complexity for scientists from diverse fields.

Looking at a sequence of zeros and ones, we often feel that it is not random, that is, it is not plausible as an outcome of fair coin tossing. Why? The answer is provided by algorithmic information theory: because the sequence is compressible, that is, it has small complexity or, equivalently, can be produced by a short program. This idea, going back to Solomonoff, Kolmogorov, Chaitin, Levin, and others, is now the starting point of algorithmic information theory. The first part of this book is a textbook-style exposition of the basic notions of complexity and randomness; the second part covers some recent work done by participants of the “Kolmogorov seminar” in Moscow (started by Kolmogorov himself in the 1980s) and their colleagues. This book contains numerous exercises (embedded in the text) that will help readers to grasp the material.

This book shows new directions in group theory motivated by computer science. It reflects the transition from geometric group theory to group theory of the 21st century that has strong connections to computer science. Now that geometric group theory is drifting further and further away from group theory to geometry, it is natural to look for new tools and new directions in group theory which are present.

The book is a collection of papers written by a selection of eminent authors from around the world in honour of Gregory Chaitin's 60th birthday. This is a unique volume including technical contributions, philosophical papers and essays.

Briefly, we review the basic elements of computability theory and prob ability theory that are required. Finally, in order to place the subject in the appropriate historical and conceptual context we trace the main roots of Kolmogorov complexity. This way the stage is set for Chapters 2 and 3, where we introduce the notion of optimal effective descriptions of objects. The length of such a description (or the number of bits of information in it) is its Kolmogorov complexity. We treat all aspects of the elementary mathematical theory of Kolmogorov complexity. This body of knowledge may be called algo rithmic complexity theory. The theory of Martin-Lof tests for random ness of finite objects and infinite sequences is inextricably intertwined with the theory of Kolmogorov complexity and is completely treated. We also investigate the statistical properties of finite strings with high Kolmogorov complexity. Both of these topics are eminently useful in the applications part of the book. We also investigate the recursion theoretic properties of Kolmogorov complexity (relations with Godel's incompleteness result), and the Kolmogorov complexity version of infor mation theory, which we may call "algorithmic information theory" or "absolute information theory. " The treatment of algorithmic probability theory in Chapter 4 presup poses Sections 1. 6, 1. 11. 2, and Chapter 3 (at least Sections 3. 1 through 3. 4).

This textbook integrates traditional statistical data analysis with new computational experimentation capabilities and concepts of algorithmic complexity and chaotic behavior in nonlinear dynamic systems. This was the first advanced text/reference to bring together such a comprehensive variety of tools for the study of random phenomena occurring in engineering and the natural, life, and social sciences. The crucial computer experiments are conducted using the readily available computer program Mathematica® Uncertain Virtual WorldsTM software packages which optimize and facilitate the simulation environment. Brief tutorials are included that explain how to use the Mathematica® programs for effective simulation and computer experiments. Large and original real-life data sets are introduced and analyzed as a model for independent study. This is an excellent classroom tool and self-study guide. The material is presented in a clear and accessible style providing numerous exercises and bibliographical notes suggesting further reading. Topics and Features Comprehensive and integrated treatment of uncertainty arising in engineering and scientific phenomena – algorithmic complexity, statistical independence, and nonlinear chaotic behavior Extensive exercise sets, examples, and Mathematica® computer experiments that reinforce concepts and algorithmic methods Thorough presentation of methods of data compression and representation Algorithmic approach to model selection and design of experiments Large data sets and 13 Mathematica®-based Uncertain Virtual WorldsTM programs and code This text is an excellent resource for all applied statisticians, engineers, and scientists who need to use modern statistical analysis methods to investigate and model their data. The present, softcover reprint is designed to make this classic textbook available to a wider audience.

This book contains a revised version of the dissertation the author wrote at the Department of Computer Science of the University of Chicago. The thesis was submitted to the Faculty of Physical Sciences in conformity with the requirements for the PhD degree in June 1999. It was honored with the 1999 ACM Doctoral Dissertation Award in May 2000. Summary Computational complexity is the study of the inherent di culty of compu- tional problems and the power of the tools we may use to solve them. It aims to describe how many resources we need to compute the solution as a function of the problem size. Typical resources include time on sequential and parallel architectures and memory space. As we want to abstract away from details of input representation and speci cs of the computer model, we end up with classes of problems that we can solve within certain robust resource bounds such as polynomial time, parallel logarithmic time, and logarithmic space. Research in complexity theory boils down to determining the relationships between these classes { inclusions and separations. In this dissertation, we focus on the role of randomness and look at various properties of hard problems in order to obtain separations. We also investigate the power of nondeterminism and alternation, as well as space versus time issues. Randomness provides a resource that seems to help in various situations.

"Algorithmic information theory (AIT) is the result of putting Shannon's information theory and Turing's computability theory into a cocktail shaker and shaking vigorously", says G.J. Chaitin, one of the fathers of this theory of complexity and randomness, which is also known as Kolmogorov complexity. It is relevant for logic (new light is shed on Gödel's incompleteness results), physics (chaotic motion), biology (how likely is life to appear and evolve?), and metaphysics (how ordered is the universe?). This book, benefiting from the author's research and teaching experience in Algorithmic Information Theory (AIT), should help to make the detailed mathematical techniques of AIT accessible to a much wider audience.