Complexities 1

“If you liked Chaos, you’ll love Complexity. Waldrop creates the most exciting intellectual adventure story of the year” (The Washington Post). In a rarified world of scientific research, a revolution has been brewing. Its activists are not anarchists, but rather Nobel Laureates in physics and economics and pony-tailed graduates, mathematicians, and computer scientists from all over the world. They have formed an iconoclastic think-tank and their radical idea is to create a new science: complexity. They want to know how a primordial soup of simple molecules managed to turn itself into the first living cell—and what the origin of life some four billion years ago can tell us about the process of technological innovation today. This book is their story—the story of how they have tried to forge what they like to call the science of the twenty-first century. “Lucidly shows physicists, biologists, computer scientists and economists swapping metaphors and reveling in the sense that epochal discoveries are just around the corner . . . [Waldrop] has a special talent for relaying the exhilaration of moments of intellectual insight.” —The New York Times Book Review “Where I enjoyed the book was when it dove into the actual question of complexity, talking about complex systems in economics, biology, genetics, computer modeling, and so on. Snippets of rare beauty here and there almost took your breath away.” —Medium “[Waldrop] provides a good grounding of what may indeed be the first flowering of a new science.” —Publishers Weekly

This book constitutes the refereed proceedings of the 14th International Colloquium on Structural Information and Communication Complexity, SIROCCO 2007, held in Castiglioncello, Italy in June 2007. The 23 revised full papers and four invited talks cover graph exploration, fault tolerance, distributed algorithms and data structures, location problems, wireless networks, fault tolerance, as well as parallel computing and selfish routing.

Briefly, we review the basic elements of computability theory and prob ability theory that are required. Finally, in order to place the subject in the appropriate historical and conceptual context we trace the main roots of Kolmogorov complexity. This way the stage is set for Chapters 2 and 3, where we introduce the notion of optimal effective descriptions of objects. The length of such a description (or the number of bits of information in it) is its Kolmogorov complexity. We treat all aspects of the elementary mathematical theory of Kolmogorov complexity. This body of knowledge may be called algo rithmic complexity theory. The theory of Martin-Lof tests for random ness of finite objects and infinite sequences is inextricably intertwined with the theory of Kolmogorov complexity and is completely treated. We also investigate the statistical properties of finite strings with high Kolmogorov complexity. Both of these topics are eminently useful in the applications part of the book. We also investigate the recursion theoretic properties of Kolmogorov complexity (relations with Godel's incompleteness result), and the Kolmogorov complexity version of infor mation theory, which we may call "algorithmic information theory" or "absolute information theory. " The treatment of algorithmic probability theory in Chapter 4 presup poses Sections 1. 6, 1. 11. 2, and Chapter 3 (at least Sections 3. 1 through 3. 4).

The classical theory of computation has its origins in the work of Goedel, Turing, Church, and Kleene and has been an extraordinarily successful framework for theoretical computer science. The thesis of this book, however, is that it provides an inadequate foundation for modern scientific computation where most of the algorithms are real number algorithms. The goal of this book is to develop a formal theory of computation which integrates major themes of the classical theory and which is more directly applicable to problems in mathematics, numerical analysis, and scientific computing. Along the way, the authors consider such fundamental problems as: * Is the Mandelbrot set decidable? * For simple quadratic maps, is the Julia set a halting set? * What is the real complexity of Newton's method? * Is there an algorithm for deciding the knapsack problem in a ploynomial number of steps? * Is the Hilbert Nullstellensatz intractable? * Is the problem of locating a real zero of a degree four polynomial intractable? * Is linear programming tractable over the reals? The book is divided into three parts: The first part provides an extensive introduction and then proves the fundamental NP-completeness theorems of Cook-Karp and their extensions to more general number fields as the real and complex numbers. The later parts of the book develop a formal theory of computation which integrates major themes of the classical theory and which is more directly applicable to problems in mathematics, numerical analysis, and scientific computing.

This book describes how model selection and statistical inference can be founded on the shortest code length for the observed data, called the stochastic complexity. This generalization of the algorithmic complexity not only offers an objective view of statistics, where no prejudiced assumptions of 'true' data generating distributions are needed, but it also in one stroke leads to calculable expressions in a range of situations of practical interest and links very closely with mainstream statistical theory. The search for the smallest stochastic complexity extends the classical maximum likelihood technique to a new global one, in which models can be compared regardless of their numbers of parameters. The result is a natural and far reaching extension of the traditional theory of estimation, where the Fisher information is replaced by the stochastic complexity and the Cramer-Rao inequality by an extension of the Shannon-Kullback inequality. Ideas are illustrated with applications from parametric and non-parametric regression, density and spectrum estimation, time series, hypothesis testing, contingency tables, and data compression.

This book constitutes the refereed proceedings of the 21st International Colloquium on Structural Information and Communication Complexity, SIROCCO 2014, held in Takayama, Japan, in July 2014. The 24 full papers presented together with 5 invited talks were carefully reviewed and selected from 51 submissions. The focus of the colloquium is on following subjects Shared Memory and Multiparty Communication, Network Optimization, CONGEST Algorithms and Lower Bounds, Wireless networks, Aggregation and Creation Games in Networks, Patrolling and Barrier Coverage, Exploration, Rendevous and Mobile Agents.

A popular way to assess the “effort” needed to solve a problem is to count how many evaluations of the problem functions (and their derivatives) are required. In many cases, this is often the dominating computational cost. Given an optimization problem satisfying reasonable assumptions—and given access to problem-function values and derivatives of various degrees—how many evaluations might be required to approximately solve the problem? Evaluation Complexity of Algorithms for Nonconvex Optimization: Theory, Computation, and Perspectives addresses this question for nonconvex optimization problems, those that may have local minimizers and appear most often in practice. This is the first book on complexity to cover topics such as composite and constrained optimization, derivative-free optimization, subproblem solution, and optimal (lower and sharpness) bounds for nonconvex problems. It is also the first to address the disadvantages of traditional optimality measures and propose useful surrogates leading to algorithms that compute approximate high-order critical points, and to compare traditional and new methods, highlighting the advantages of the latter from a complexity point of view. This is the go-to book for those interested in solving nonconvex optimization problems. It is suitable for advanced undergraduate and graduate students in courses on advanced numerical analysis, data science, numerical optimization, and approximation theory.