A Probabilistic Theory Of Pattern Recognition

A self-contained and coherent account of probabilistic techniques, covering: distance measures, kernel rules, nearest neighbour rules, Vapnik-Chervonenkis theory, parametric classification, and feature extraction. Each chapter concludes with problems and exercises to further the readers understanding. Both research workers and graduate students will benefit from this wide-ranging and up-to-date account of a fast- moving field.

This is the first textbook on pattern recognition to present the Bayesian viewpoint. The book presents approximate inference algorithms that permit fast approximate answers in situations where exact answers are not feasible. It uses graphical models to describe probability distributions when no other books apply graphical models to machine learning. No previous knowledge of pattern recognition or machine learning concepts is assumed. Familiarity with multivariate calculus and basic linear algebra is required, and some experience in the use of probabilities would be helpful though not essential as the book includes a self-contained introduction to basic probability theory.

Statistical pattern recognition; Probability density estimation; Single-layer networks; The multi-layer perceptron; Radial basis functions; Error functions; Parameter optimization algorithms; Pre-processing and feature extraction; Learning and generalization; Bayesian techniques; Appendix; References; Index.

This volume provides a systematic in-depth analysis of nonparametric learning. It covers the theoretical limits and the asymptotical optimal algorithms and estimates, such as pattern recognition, nonparametric regression estimation, universal prediction, vector quantization, distribution and density estimation, and genetic programming.

A comprehensive introduction to machine learning that uses probabilistic models and inference as a unifying approach. Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package—PMTK (probabilistic modeling toolkit)—that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.

A general framework for constructing and using probabilistic models of complex systems that would enable a computer to use available information for making decisions. Most tasks require a person or an automated system to reason—to reach conclusions based on available information. The framework of probabilistic graphical models, presented in this book, provides a general approach for this task. The approach is model-based, allowing interpretable models to be constructed and then manipulated by reasoning algorithms. These models can also be learned automatically from data, allowing the approach to be used in cases where manually constructing a model is difficult or even impossible. Because uncertainty is an inescapable aspect of most real-world applications, the book focuses on probabilistic models, which make the uncertainty explicit and provide models that are more faithful to reality. Probabilistic Graphical Models discusses a variety of models, spanning Bayesian networks, undirected Markov networks, discrete and continuous models, and extensions to deal with dynamical systems and relational data. For each class of models, the text describes the three fundamental cornerstones: representation, inference, and learning, presenting both basic concepts and advanced techniques. Finally, the book considers the use of the proposed framework for causal reasoning and decision making under uncertainty. The main text in each chapter provides the detailed technical development of the key ideas. Most chapters also include boxes with additional material: skill boxes, which describe techniques; case study boxes, which discuss empirical cases related to the approach described in the text, including applications in computer vision, robotics, natural language understanding, and computational biology; and concept boxes, which present significant concepts drawn from the material in the chapter. Instructors (and readers) can group chapters in various combinations, from core topics to more technically advanced material, to suit their particular needs.

Introduction to Probabilistic Automata deals with stochastic sequential machines, Markov chains, events, languages, acceptors, and applications. The book describes mathematical models of stochastic sequential machines (SSMs), stochastic input-output relations, and their representation by SSMs. The text also investigates decision problems and minimization-of-states problems arising from concepts of equivalence and coverings for SSMs. The book presents the theory of nonhomogeneous Markov chains and systems in mathematical terms, particularly in relation to asymptotic behavior, composition (direct sum or product), and decomposition. "Word functions," induced by Markov chains and valued Markov systems, involve characterization, equivalence, and representability by an underlying Markov chain or system. The text also discusses the closure properties of probabilistic languages, events and their relation to regular events, particularly with reference to definite, quasidefinite, and exclusive events. Probabilistic automata theory has applications in information theory, control, learning theory, pattern recognition, and time sharing in computer programming. Programmers, computer engineers, computer instructors, and students of computer science will find the collection highly valuable.