Information And Complexity In Statistical Modeling

No statistical model is "true" or "false," "right" or "wrong"; the models just have varying performance, which can be assessed. The main theme in this book is to teach modeling based on the principle that the objective is to extract the information from data that can be learned with suggested classes of probability models. The intuitive and fundamental concepts of complexity, learnable information, and noise are formalized, which provides a firm information theoretic foundation for statistical modeling. Although the prerequisites include only basic probability calculus and statistics, a moderate level of mathematical proficiency would be beneficial.

This book reviews some of today’s more complex problems, and reflects some of the important research directions in the field. Twenty-nine authors – largely from Montreal’s GERAD Multi-University Research Center and who work in areas of theoretical statistics, applied statistics, probability theory, and stochastic processes – present survey chapters on various theoretical and applied problems of importance and interest to researchers and students across a number of academic domains.

The book is addressed to statisticians working at the forefront of the statistical analysis of complex and high dimensional data and offers a wide variety of statistical models, computer intensive methods and applications: network inference from the analysis of high dimensional data; new developments for bootstrapping complex data; regression analysis for measuring the downsize reputational risk; statistical methods for research on the human genome dynamics; inference in non-euclidean settings and for shape data; Bayesian methods for reliability and the analysis of complex data; methodological issues in using administrative data for clinical and epidemiological research; regression models with differential regularization; geostatistical methods for mobility analysis through mobile phone data exploration. This volume is the result of a careful selection among the contributions presented at the conference "S.Co.2013: Complex data modeling and computationally intensive methods for estimation and prediction" held at the Politecnico di Milano, 2013. All the papers published here have been rigorously peer-reviewed.

This book describes how model selection and statistical inference can be founded on the shortest code length for the observed data, called the stochastic complexity. This generalization of the algorithmic complexity not only offers an objective view of statistics, where no prejudiced assumptions of 'true' data generating distributions are needed, but it also in one stroke leads to calculable expressions in a range of situations of practical interest and links very closely with mainstream statistical theory. The search for the smallest stochastic complexity extends the classical maximum likelihood technique to a new global one, in which models can be compared regardless of their numbers of parameters. The result is a natural and far reaching extension of the traditional theory of estimation, where the Fisher information is replaced by the stochastic complexity and the Cramer-Rao inequality by an extension of the Shannon-Kullback inequality. Ideas are illustrated with applications from parametric and non-parametric regression, density and spectrum estimation, time series, hypothesis testing, contingency tables, and data compression.

Selected from the conference "S.Co.2009: Complex Data Modeling and Computationally Intensive Methods for Estimation and Prediction," these 20 papers cover the latest in statistical methods and computational techniques for complex and high dimensional datasets.

Several points of disagreement exist between different modelling traditions as to whether complex models are always better than simpler models, as to how to combine results from different models and how to propagate model uncertainty into forecasts. This book represents the result of collaboration between scientists from many disciplines to show how these conflicts can be resolved. Key Features: Introduces important concepts in modelling, outlining different traditions in the use of simple and complex modelling in statistics. Provides numerous case studies on complex modelling, such as climate change, flood risk and new drug development. Concentrates on varying models, including flood risk analysis models, the petrol industry forecasts and summarizes the evolution of water distribution systems. Written by experienced statisticians and engineers in order to facilitate communication between modellers in different disciplines. Provides a glossary giving terms commonly used in different modelling traditions. This book provides a much-needed reference guide to approaching statistical modelling. Scientists involved with modelling complex systems in areas such as climate change, flood prediction and prevention, financial market modelling and systems engineering will benefit from this book. It will also be a useful source of modelling case histories.

This textbook introduces a science philosophy called "information theoretic" based on Kullback-Leibler information theory. It focuses on a science philosophy based on "multiple working hypotheses" and statistical models to represent them. The text is written for people new to the information-theoretic approaches to statistical inference, whether graduate students, post-docs, or professionals. Readers are however expected to have a background in general statistical principles, regression analysis, and some exposure to likelihood methods. This is not an elementary text as it assumes reasonable competence in modeling and parameter estimation.

This introduction to the MDL Principle provides a reference accessible to graduate students and researchers in statistics, pattern classification, machine learning, and data mining, to philosophers interested in the foundations of statistics, and to researchers in other applied sciences that involve model selection.