Probably Approximately Correct Learning

Abstract: "This paper surveys some recent theoretical results on the efficiency of machine learning algorithms. The main tool described is the notion of Probably Approximately Correct (PAC) learning, introduced by Valiant. We define this learning model and then look at some of the results obtained in it. We then consider some criticisms of the PAC model and the extensions proposed to address these criticisms. Finally, we look briefly at other models recently proposed in computational learning theory."

We have effective theories for very few things. Gravity is one, electromagnetism another. But for most things—whether as mundane as finding a mate or as major as managing an economy—our theories are lousy or nonexistent. Fortunately, we don't need them, any more than a fish needs a theory of water to swim; we're able to muddle through. But how do we do it? In Probably Approximately Correct, computer scientist Leslie Valiant presents a theory of the theoryless. The key is “probably approximately correct” learning, Valiant's model of how anything can act without needing to understand what is going on. The study of probably approximately correct algorithms reveals the shared computational nature of evolution and cognition, indicates how computers might possess authentic intelligence, and shows why hacking a problem can be far more effective than developing a theory to explain it. After all, finding a mate is a lot more satisfying than finding a theory of mating. Offering an elegant, powerful model that encompasses all of life's complexity, Probably Approximately Correct will revolutionize the way we look at the universe's greatest mysteries.

Presenting a theory of the theoryless, a computer scientist provides a model of how effective behavior can be learned even in a world as complex as our own, shedding new light on human nature.

From a leading computer scientist, a unifying theory that will revolutionize our understanding of how life evolves and learns. How does life prosper in a complex and erratic world? While we know that nature follows patterns -- such as the law of gravity -- our everyday lives are beyond what known science can predict. We nevertheless muddle through even in the absence of theories of how to act. But how do we do it? In Probably Approximately Correct, computer scientist Leslie Valiant presents a masterful synthesis of learning and evolution to show how both individually and collectively we not only survive, but prosper in a world as complex as our own. The key is "probably approximately correct" algorithms, a concept Valiant developed to explain how effective behavior can be learned. The model shows that pragmatically coping with a problem can provide a satisfactory solution in the absence of any theory of the problem. After all, finding a mate does not require a theory of mating. Valiant's theory reveals the shared computational nature of evolution and learning, and sheds light on perennial questions such as nature versus nurture and the limits of artificial intelligence. Offering a powerful and elegant model that encompasses life's complexity, Probably Approximately Correct has profound implications for how we think about behavior, cognition, biological evolution, and the possibilities and limits of human and machine intelligence.

Emphasizing issues of computational efficiency, Michael Kearns and Umesh Vazirani introduce a number of central topics in computational learning theory for researchers and students in artificial intelligence, neural networks, theoretical computer science, and statistics. Emphasizing issues of computational efficiency, Michael Kearns and Umesh Vazirani introduce a number of central topics in computational learning theory for researchers and students in artificial intelligence, neural networks, theoretical computer science, and statistics. Computational learning theory is a new and rapidly expanding area of research that examines formal models of induction with the goals of discovering the common methods underlying efficient learning algorithms and identifying the computational impediments to learning. Each topic in the book has been chosen to elucidate a general principle, which is explored in a precise formal setting. Intuition has been emphasized in the presentation to make the material accessible to the nontheoretician while still providing precise arguments for the specialist. This balance is the result of new proofs of established theorems, and new presentations of the standard proofs. The topics covered include the motivation, definitions, and fundamental results, both positive and negative, for the widely studied L. G. Valiant model of Probably Approximately Correct Learning; Occam's Razor, which formalizes a relationship between learning and data compression; the Vapnik-Chervonenkis dimension; the equivalence of weak and strong learning; efficient learning in the presence of noise by the method of statistical queries; relationships between learning and cryptography, and the resulting computational limitations on efficient learning; reducibility between learning problems; and algorithms for learning finite automata from active experimentation.

Introduces machine learning and its algorithmic paradigms, explaining the principles behind automated learning approaches and the considerations underlying their usage.

Table of contents

One of Springer’s renowned Major Reference Works, this awesome achievement provides a comprehensive set of solutions to important algorithmic problems for students and researchers interested in quickly locating useful information. This first edition of the reference focuses on high-impact solutions from the most recent decade, while later editions will widen the scope of the work. All entries have been written by experts, while links to Internet sites that outline their research work are provided. The entries have all been peer-reviewed. This defining reference is published both in print and on line.