Complexities 1

This book constitutes the refereed proceedings of the 14th International Colloquium on Structural Information and Communication Complexity, SIROCCO 2007, held in Castiglioncello, Italy in June 2007. The 23 revised full papers and four invited talks cover graph exploration, fault tolerance, distributed algorithms and data structures, location problems, wireless networks, fault tolerance, as well as parallel computing and selfish routing.

“If you liked Chaos, you’ll love Complexity. Waldrop creates the most exciting intellectual adventure story of the year” (The Washington Post). In a rarified world of scientific research, a revolution has been brewing. Its activists are not anarchists, but rather Nobel Laureates in physics and economics and pony-tailed graduates, mathematicians, and computer scientists from all over the world. They have formed an iconoclastic think-tank and their radical idea is to create a new science: complexity. They want to know how a primordial soup of simple molecules managed to turn itself into the first living cell—and what the origin of life some four billion years ago can tell us about the process of technological innovation today. This book is their story—the story of how they have tried to forge what they like to call the science of the twenty-first century. “Lucidly shows physicists, biologists, computer scientists and economists swapping metaphors and reveling in the sense that epochal discoveries are just around the corner . . . [Waldrop] has a special talent for relaying the exhilaration of moments of intellectual insight.” —The New York Times Book Review “Where I enjoyed the book was when it dove into the actual question of complexity, talking about complex systems in economics, biology, genetics, computer modeling, and so on. Snippets of rare beauty here and there almost took your breath away.” —Medium “[Waldrop] provides a good grounding of what may indeed be the first flowering of a new science.” —Publishers Weekly

Briefly, we review the basic elements of computability theory and prob ability theory that are required. Finally, in order to place the subject in the appropriate historical and conceptual context we trace the main roots of Kolmogorov complexity. This way the stage is set for Chapters 2 and 3, where we introduce the notion of optimal effective descriptions of objects. The length of such a description (or the number of bits of information in it) is its Kolmogorov complexity. We treat all aspects of the elementary mathematical theory of Kolmogorov complexity. This body of knowledge may be called algo rithmic complexity theory. The theory of Martin-Lof tests for random ness of finite objects and infinite sequences is inextricably intertwined with the theory of Kolmogorov complexity and is completely treated. We also investigate the statistical properties of finite strings with high Kolmogorov complexity. Both of these topics are eminently useful in the applications part of the book. We also investigate the recursion theoretic properties of Kolmogorov complexity (relations with Godel's incompleteness result), and the Kolmogorov complexity version of infor mation theory, which we may call "algorithmic information theory" or "absolute information theory. " The treatment of algorithmic probability theory in Chapter 4 presup poses Sections 1. 6, 1. 11. 2, and Chapter 3 (at least Sections 3. 1 through 3. 4).

This book describes how model selection and statistical inference can be founded on the shortest code length for the observed data, called the stochastic complexity. This generalization of the algorithmic complexity not only offers an objective view of statistics, where no prejudiced assumptions of 'true' data generating distributions are needed, but it also in one stroke leads to calculable expressions in a range of situations of practical interest and links very closely with mainstream statistical theory. The search for the smallest stochastic complexity extends the classical maximum likelihood technique to a new global one, in which models can be compared regardless of their numbers of parameters. The result is a natural and far reaching extension of the traditional theory of estimation, where the Fisher information is replaced by the stochastic complexity and the Cramer-Rao inequality by an extension of the Shannon-Kullback inequality. Ideas are illustrated with applications from parametric and non-parametric regression, density and spectrum estimation, time series, hypothesis testing, contingency tables, and data compression.

This book constitutes the refereed proceedings of the 21st International Colloquium on Structural Information and Communication Complexity, SIROCCO 2014, held in Takayama, Japan, in July 2014. The 24 full papers presented together with 5 invited talks were carefully reviewed and selected from 51 submissions. The focus of the colloquium is on following subjects Shared Memory and Multiparty Communication, Network Optimization, CONGEST Algorithms and Lower Bounds, Wireless networks, Aggregation and Creation Games in Networks, Patrolling and Barrier Coverage, Exploration, Rendevous and Mobile Agents.

A popular way to assess the “effort” needed to solve a problem is to count how many evaluations of the problem functions (and their derivatives) are required. In many cases, this is often the dominating computational cost. Given an optimization problem satisfying reasonable assumptions—and given access to problem-function values and derivatives of various degrees—how many evaluations might be required to approximately solve the problem? Evaluation Complexity of Algorithms for Nonconvex Optimization: Theory, Computation, and Perspectives addresses this question for nonconvex optimization problems, those that may have local minimizers and appear most often in practice. This is the first book on complexity to cover topics such as composite and constrained optimization, derivative-free optimization, subproblem solution, and optimal (lower and sharpness) bounds for nonconvex problems. It is also the first to address the disadvantages of traditional optimality measures and propose useful surrogates leading to algorithms that compute approximate high-order critical points, and to compare traditional and new methods, highlighting the advantages of the latter from a complexity point of view. This is the go-to book for those interested in solving nonconvex optimization problems. It is suitable for advanced undergraduate and graduate students in courses on advanced numerical analysis, data science, numerical optimization, and approximation theory.

A clear, comprehensive, and rigorous introduction to the theory of computation. What is computable? What leads to efficiency in computation? Computability and Complexity offers a clear, comprehensive, and rigorous introduction to the mathematical study of the capabilities and limitations of computation. Hubie Chen covers the core notions, techniques, methods, and questions of the theory of computation before turning to several advanced topics. Emphasizing intuitive learning and conceptual discussion, this textbook’s accessible approach offers a robust foundation for understanding both the reach and restrictions of algorithms and computers. Extensive exercises and diagrams enhance streamlined, student-friendly presentation of mathematically rigorous material Includes thorough treatment of automata theory, computability theory, and complexity theory—including the P versus NP question and the theory of NP-completeness Suitable for undergraduate and graduate students, researchers, and professionals