Information And Complexity In Statistical Modeling

No statistical model is "true" or "false," "right" or "wrong"; the models just have varying performance, which can be assessed. The main theme in this book is to teach modeling based on the principle that the objective is to extract the information from data that can be learned with suggested classes of probability models. The intuitive and fundamental concepts of complexity, learnable information, and noise are formalized, which provides a firm information theoretic foundation for statistical modeling. Although the prerequisites include only basic probability calculus and statistics, a moderate level of mathematical proficiency would be beneficial.

This book describes how model selection and statistical inference can be founded on the shortest code length for the observed data, called the stochastic complexity. This generalization of the algorithmic complexity not only offers an objective view of statistics, where no prejudiced assumptions of 'true' data generating distributions are needed, but it also in one stroke leads to calculable expressions in a range of situations of practical interest and links very closely with mainstream statistical theory. The search for the smallest stochastic complexity extends the classical maximum likelihood technique to a new global one, in which models can be compared regardless of their numbers of parameters. The result is a natural and far reaching extension of the traditional theory of estimation, where the Fisher information is replaced by the stochastic complexity and the Cramer-Rao inequality by an extension of the Shannon-Kullback inequality. Ideas are illustrated with applications from parametric and non-parametric regression, density and spectrum estimation, time series, hypothesis testing, contingency tables, and data compression.

This book reviews some of today’s more complex problems, and reflects some of the important research directions in the field. Twenty-nine authors – largely from Montreal’s GERAD Multi-University Research Center and who work in areas of theoretical statistics, applied statistics, probability theory, and stochastic processes – present survey chapters on various theoretical and applied problems of importance and interest to researchers and students across a number of academic domains.

Selected from the conference "S.Co.2009: Complex Data Modeling and Computationally Intensive Methods for Estimation and Prediction," these 20 papers cover the latest in statistical methods and computational techniques for complex and high dimensional datasets.

Several points of disagreement exist between different modelling traditions as to whether complex models are always better than simpler models, as to how to combine results from different models and how to propagate model uncertainty into forecasts. This book represents the result of collaboration between scientists from many disciplines to show how these conflicts can be resolved. Key Features: Introduces important concepts in modelling, outlining different traditions in the use of simple and complex modelling in statistics. Provides numerous case studies on complex modelling, such as climate change, flood risk and new drug development. Concentrates on varying models, including flood risk analysis models, the petrol industry forecasts and summarizes the evolution of water distribution systems. Written by experienced statisticians and engineers in order to facilitate communication between modellers in different disciplines. Provides a glossary giving terms commonly used in different modelling traditions. This book provides a much-needed reference guide to approaching statistical modelling. Scientists involved with modelling complex systems in areas such as climate change, flood prediction and prevention, financial market modelling and systems engineering will benefit from this book. It will also be a useful source of modelling case histories.

Statistical Foundations of Data Science gives a thorough introduction to commonly used statistical models, contemporary statistical machine learning techniques and algorithms, along with their mathematical insights and statistical theories. It aims to serve as a graduate-level textbook and a research monograph on high-dimensional statistics, sparsity and covariance learning, machine learning, and statistical inference. It includes ample exercises that involve both theoretical studies as well as empirical applications. The book begins with an introduction to the stylized features of big data and their impacts on statistical analysis. It then introduces multiple linear regression and expands the techniques of model building via nonparametric regression and kernel tricks. It provides a comprehensive account on sparsity explorations and model selections for multiple regression, generalized linear models, quantile regression, robust regression, hazards regression, among others. High-dimensional inference is also thoroughly addressed and so is feature screening. The book also provides a comprehensive account on high-dimensional covariance estimation, learning latent factors and hidden structures, as well as their applications to statistical estimation, inference, prediction and machine learning problems. It also introduces thoroughly statistical machine learning theory and methods for classification, clustering, and prediction. These include CART, random forests, boosting, support vector machines, clustering algorithms, sparse PCA, and deep learning.

The understanding of electron density as the carrier of all the information of a multielectronic system is implicit in the theorems of density functional theory. Information theoretical based measures giving a quantitative understanding of statistical complexity of such systems is shaping up as a new area of research in chemical physics. This book is the first monograph of its kind covering the aspects of complexity measure in atoms and molecules.

The book is addressed to statisticians working at the forefront of the statistical analysis of complex and high dimensional data and offers a wide variety of statistical models, computer intensive methods and applications: network inference from the analysis of high dimensional data; new developments for bootstrapping complex data; regression analysis for measuring the downsize reputational risk; statistical methods for research on the human genome dynamics; inference in non-euclidean settings and for shape data; Bayesian methods for reliability and the analysis of complex data; methodological issues in using administrative data for clinical and epidemiological research; regression models with differential regularization; geostatistical methods for mobility analysis through mobile phone data exploration. This volume is the result of a careful selection among the contributions presented at the conference "S.Co.2013: Complex data modeling and computationally intensive methods for estimation and prediction" held at the Politecnico di Milano, 2013. All the papers published here have been rigorously peer-reviewed.