Bayesian Natural Language Semantics And Pragmatics

The contributions in this volume focus on the Bayesian interpretation of natural languages, which is widely used in areas of artificial intelligence, cognitive science, and computational linguistics. This is the first volume to take up topics in Bayesian Natural Language Interpretation and make proposals based on information theory, probability theory, and related fields. The methodologies offered here extend to the target semantic and pragmatic analyses of computational natural language interpretation. Bayesian approaches to natural language semantics and pragmatics are based on methods from signal processing and the causal Bayesian models pioneered by especially Pearl. In signal processing, the Bayesian method finds the most probable interpretation by finding the one that maximizes the product of the prior probability and the likelihood of the interpretation. It thus stresses the importance of a production model for interpretation as in Grice’s contributions to pragmatics or in interpretation by abduction.

Natural language processing (NLP) went through a profound transformation in the mid-1980s when it shifted to make heavy use of corpora and data-driven techniques to analyze language. Since then, the use of statistical techniques in NLP has evolved in several ways. One such example of evolution took place in the late 1990s or early 2000s, when full-fledged Bayesian machinery was introduced to NLP. This Bayesian approach to NLP has come to accommodate for various shortcomings in the frequentist approach and to enrich it, especially in the unsupervised setting, where statistical learning is done without target prediction examples. We cover the methods and algorithms that are needed to fluently read Bayesian learning papers in NLP and to do research in the area. These methods and algorithms are partially borrowed from both machine learning and statistics and are partially developed "in-house" in NLP. We cover inference techniques such as Markov chain Monte Carlo sampling and variational inference, Bayesian estimation, and nonparametric modeling. We also cover fundamental concepts in Bayesian statistics such as prior distributions, conjugacy, and generative modeling. Finally, we cover some of the fundamental modeling techniques in NLP, such as grammar modeling and their use with Bayesian analysis.

An utterance is normally produced by a speaker in linear time and the hearer normally correctly identifies the speaker intention in linear time and incrementally. This is hard to understand in a standard competence grammar since languages are highly ambiguous and context-free parsing is not linear. Deterministic utterance generation from intention and n-best Bayesian interpretation, based on the production grammar and the prior probabilities that need to be assumed for other perception do much better. The proposed model uses symbolic grammar and derives symbolic semantic representations, but treats interpretation as just another form of perception. Removing interpretation from grammar is not only empirically motivated, but also makes linguistics a much more feasible enterprise. The importance of Henk Zeevat's new monograph cannot be overstated. Its combination of breadth, formal rigor, and originality is unparalleled in work on the form-meaning interface in human language...Zeevat's is the first proposal which provides a computationally feasible integrated treatment of production and comprehension for pragmatics, semantics, syntax, and even phonology. I recommend it to anyone who combines interests in language, logic, and computation with a sense of adventure. David Beaver, University of Texas at Austin

Natural language processing (NLP) went through a profound transformation in the mid-1980s when it shifted to make heavy use of corpora and data-driven techniques to analyze language. Since then, the use of statistical techniques in NLP has evolved in several ways. One such example of evolution took place in the late 1990s or early 2000s, when full-fledged Bayesian machinery was introduced to NLP. This Bayesian approach to NLP has come to accommodate various shortcomings in the frequentist approach and to enrich it, especially in the unsupervised setting, where statistical learning is done without target prediction examples. In this book, we cover the methods and algorithms that are needed to fluently read Bayesian learning papers in NLP and to do research in the area. These methods and algorithms are partially borrowed from both machine learning and statistics and are partially developed "in-house" in NLP. We cover inference techniques such as Markov chain Monte Carlo sampling and variational inference, Bayesian estimation, and nonparametric modeling. In response to rapid changes in the field, this second edition of the book includes a new chapter on representation learning and neural networks in the Bayesian context. We also cover fundamental concepts in Bayesian statistics such as prior distributions, conjugacy, and generative modeling. Finally, we review some of the fundamental modeling techniques in NLP, such as grammar modeling, neural networks and representation learning, and their use with Bayesian analysis.

Natural language processing (NLP) went through a profound transformation in the mid-1980s when it shifted to make heavy use of corpora and data-driven techniques to analyze language. Since then, the use of statistical techniques in NLP has evolved in several ways. One such example of evolution took place in the late 1990s or early 2000s, when full-fledged Bayesian machinery was introduced to NLP. This Bayesian approach to NLP has come to accommodate various shortcomings in the frequentist approach and to enrich it, especially in the unsupervised setting, where statistical learning is done without target prediction examples. In this book, we cover the methods and algorithms that are needed to fluently read Bayesian learning papers in NLP and to do research in the area. These methods and algorithms are partially borrowed from both machine learning and statistics and are partially developed "in-house" in NLP. We cover inference techniques such as Markov chain Monte Carlo sampling and variational inference, Bayesian estimation, and nonparametric modeling. In response to rapid changes in the field, this second edition of the book includes a new chapter on representation learning and neural networks in the Bayesian context. We also cover fundamental concepts in Bayesian statistics such as prior distributions, conjugacy, and generative modeling. Finally, we review some of the fundamental modeling techniques in NLP, such as grammar modeling, neural networks and representation learning, and their use with Bayesian analysis.

What is the remit of theoretical linguistics? How are human languages different from animal calls or artificial languages? What philosophical insights about language can be gleaned from phonology, pragmatics, probabilistic linguistics, and deep learning? This book addresses the current philosophical issues at the heart of theoretical linguistics, which are widely debated not only by linguists, but also philosophers, psychologists, and computer scientists. It delves into hitherto uncharted territory, putting philosophy in direct conversation with phonology, sign language studies, supersemantics, computational linguistics, and language evolution. A range of theoretical positions are covered, from optimality theory and autosegmental phonology to generative syntax, dynamic semantics, and natural language processing with deep learning techniques. By both unwinding the complexities of natural language and delving into the nature of the science that studies it, this book ultimately improves our tools of discovery aimed at one of the most essential features of our humanity, our language.

This handbook presents an overview of the phenomenon of reference - the ability to refer to and pick out entities - which is an essential part of human language and cognition. In the volume's 21 chapters, international experts in the field offer a critical account of all aspects of reference from a range of theoretical perspectives. Chapters in the first part of the book are concerned with basic questions related to different types of referring expression and their interpretation. They address questions about the role of the speaker - including speaker intentions - and of the addressee, as well as the role played by the semantics of the linguistic forms themselves in establishing reference. This part also explores the nature of such concepts as definite and indefinite reference and specificity, and the conditions under which reference may fail. The second part of the volume looks at implications and applications, with chapters covering such topics as the acquisition of reference by children, the processing of reference both in the human brain and by machines. The volume will be of interest to linguists in a wide range of subfields, including semantics, pragmatics, computational linguistics, and psycho- and neurolinguistics, as well as scholars in related fields such as philosophy and computer science.