Reasoning With Data

Engaging and accessible, this book teaches readers how to use inferential statistical thinking to check their assumptions, assess evidence about their beliefs, and avoid overinterpreting results that may look more promising than they really are. It provides step-by-step guidance for using both classical (frequentist) and Bayesian approaches to inference. Statistical techniques covered side by side from both frequentist and Bayesian approaches include hypothesis testing, replication, analysis of variance, calculation of effect sizes, regression, time series analysis, and more. Students also get a complete introduction to the open-source R programming language and its key packages. Throughout the text, simple commands in R demonstrate essential data analysis skills using real-data examples. The companion website provides annotated R code for the book's examples, in-class exercises, supplemental reading lists, and links to online videos, interactive materials, and other resources. ÿ Pedagogical Features *Playful, conversational style and gradual approach; suitable for students without strong math backgrounds. *End-of-chapter exercises based on real data supplied in the free R package. *Technical explanation and equation/output boxes. *Appendices on how to install R and work with the sample datasets.ÿ

An engaging introduction to data science that emphasizes critical thinking over statistical techniques An introduction to data science or statistics shouldn’t involve proving complex theorems or memorizing obscure terms and formulas, but that is exactly what most introductory quantitative textbooks emphasize. In contrast, Thinking Clearly with Data focuses, first and foremost, on critical thinking and conceptual understanding in order to teach students how to be better consumers and analysts of the kinds of quantitative information and arguments that they will encounter throughout their lives. Among much else, the book teaches how to assess whether an observed relationship in data reflects a genuine relationship in the world and, if so, whether it is causal; how to make the most informative comparisons for answering questions; what questions to ask others who are making arguments using quantitative evidence; which statistics are particularly informative or misleading; how quantitative evidence should and shouldn’t influence decision-making; and how to make better decisions by using moral values as well as data. Filled with real-world examples, the book shows how its thinking tools apply to problems in a wide variety of subjects, including elections, civil conflict, crime, terrorism, financial crises, health care, sports, music, and space travel. Above all else, Thinking Clearly with Data demonstrates why, despite the many benefits of our data-driven age, data can never be a substitute for thinking. An ideal textbook for introductory quantitative methods courses in data science, statistics, political science, economics, psychology, sociology, public policy, and other fields Introduces the basic toolkit of data analysis—including sampling, hypothesis testing, Bayesian inference, regression, experiments, instrumental variables, differences in differences, and regression discontinuity Uses real-world examples and data from a wide variety of subjects Includes practice questions and data exercises

This book provides a multi-level introduction to Bayesian reasoning (as opposed to “conventional statistics”) and its applications to data analysis. The basic ideas of this “new” approach to the quantification of uncertainty are presented using examples from research and everyday life. Applications covered include: parametric inference; combination of results; treatment of uncertainty due to systematic errors and background; comparison of hypotheses; unfolding of experimental distributions; upper/lower bounds in frontier-type measurements. Approximate methods for routine use are derived and are shown often to coincide — under well-defined assumptions! — with “standard” methods, which can therefore be seen as special cases of the more general Bayesian methods. In dealing with uncertainty in measurements, modern metrological ideas are utilized, including the ISO classification of uncertainty into type A and type B. These are shown to fit well into the Bayesian framework. Contents: Critical Review and Outline of the Bayesian Alternative:Uncertainty in Physics and the Usual Methods of Handling ItA Probabilistic Theory of Measurement UncertaintyA Bayesian Primer:Subjective Probability and Bayes' TheoremProbability Distributions (A Concise Reminder)Bayesian Inference of Continuous QuantitiesGaussian LikelihoodCounting ExperimentsBypassing Bayes' Theorem for Routine ApplicationsBayesian UnfoldingFurther Comments, Examples and Applications:Miscellanea on General Issues in Probability and InferenceCombination of Experimental Results: A Closer LookAsymmetric Uncertainties and Nonlinear PropagationWhich Priors for Frontier Physics?Conclusion:Conclusions and Bibliography Readership: Graduate students and researchers interested in probability and statistics and their applications in science, particularly the evaluation of uncertainty in measurements. Keywords:Probability;Bayesian Statistics;Error Theory;Measurement Uncertainty;MetrologyReviews:“… statistics textbooks must take seriously the need to teach the foundations of statistical reasoning from the beginning … D'Agostini's new book does this admirably, building an edifice of Bayesian statistical reasoning in the physical sciences on solid foundations.”Journal of the American Statistical Association

Linked Data publishing has brought about a novel “Web of Data”: a wealth of diverse, interlinked, structured data published on the Web. These Linked Datasets are described using the Semantic Web standards and are openly available to all, produced by governments, businesses, communities and academia alike. However, the heterogeneity of such data – in terms of how resources are described and identified – poses major challenges to potential consumers. Herein, we examine use cases for pragmatic, lightweight reasoning techniques that leverage Web vocabularies (described in RDFS and OWL) to better integrate large scale, diverse, Linked Data corpora. We take a test corpus of 1.1 billion RDF statements collected from 4 million RDF Web documents and analyse the use of RDFS and OWL therein. We then detail and evaluate scalable and distributed techniques for applying rule-based materialisation to translate data between different vocabularies, and to resolve coreferent resources that talk about the same thing. We show how such techniques can be made robust in the face of noisy and often impudent Web data. We also examine a use case for incorporating a PagerRank-style algorithm to rank the trustworthiness of facts produced by reasoning, subsequently using those ranks to fix formal contradictions in the data. All of our methods are validated against our real world, large scale, open domain, Linked Data evaluation corpus.

Many analysts are too concerned with tools and techniques for cleansing, modeling, and visualizing datasets and not concerned enough with asking the right questions. In this practical guide, data strategy consultant Max Shron shows you how to put the why before the how, through an often-overlooked set of analytical skills. Thinking with Data helps you learn techniques for turning data into knowledge you can use. You’ll learn a framework for defining your project, including the data you want to collect, and how you intend to approach, organize, and analyze the results. You’ll also learn patterns of reasoning that will help you unveil the real problem that needs to be solved. Learn a framework for scoping data projects Understand how to pin down the details of an idea, receive feedback, and begin prototyping Use the tools of arguments to ask good questions, build projects in stages, and communicate results Explore data-specific patterns of reasoning and learn how to build more useful arguments Delve into causal reasoning and learn how it permeates data work Put everything together, using extended examples to see the method of full problem thinking in action

This book constitutes the refereed proceedings of the Second International Symposium on Intelligent Data Analysis, IDA-97, held in London, UK, in August 1997. The volume presents 50 revised full papers selected from a total of 107 submissions. Also included is a keynote, Intelligent Data Analysis: Issues and Opportunities, by David J. Hand. The papers are organized in sections on exploratory data analysis, preprocessing and tools; classification and feature selection; medical applications; soft computing; knowledge discovery and data mining; estimation and clustering; data quality; qualitative models.

Unique in that it collects, presents, and synthesizes cutting edge research on different aspects of statistical reasoning and applies this research to the teaching of statistics to students at all educational levels, this volume will prove of great value to mathematics and statistics education researchers, statistics educators, statisticians, cognitive psychologists, mathematics teachers, mathematics and statistics curriculum developers, and quantitative literacy experts in education and government.

Accompanying CD-ROM contains ... "support material for many of the articles, including lessons, software demonstrations, and even video clips of classrooms."--P. [4] of cover.