The Significance Test Controversy

The purpose of this book is not only to revisit the “significance test controversy,”but also to provide a conceptually sounder alternative. As such, it presents a Bayesian framework for a new approach to analyzing and interpreting experimental data. It also prepares students and researchers for reporting on experimental results. Normative aspects: The main views of statistical tests are revisited and the philosophies of Fisher, Neyman-Pearson and Jeffrey are discussed in detail. Descriptive aspects: The misuses of Null Hypothesis Significance Tests are reconsidered in light of Jeffreys’ Bayesian conceptions concerning the role of statistical inference in experimental investigations. Prescriptive aspects: The current effect size and confidence interval reporting practices are presented and seriously questioned. Methodological aspects are carefully discussed and fiducial Bayesian methods are proposed as a more suitable alternative for reporting on experimental results. In closing, basic routine procedures regarding the means and their generalization to the most common ANOVA applications are presented and illustrated. All the calculations discussed can be easily carried out using the freeware LePAC package.

Tests of significance have been a key tool in the research kit of behavioral scientists for nearly fifty years, but their widespread and uncritical use has recently led to a rising volume of controversy about their usefulness. This book gathers the central papers in this continuing debate, brings the issues into clear focus, points out practical problems and philosophical pitfalls involved in using the tests, and provides a benchmark from which further analysis can proceed.The papers deal with some of the basic philosophy of science, mathematical and statistical assumptions connected with significance tests and the problems of the interpretation of test results, but the work is essentially non-technical in its emphasis. The collection succeeds in raising a variety of questions about the value of the tests; taken together, the questions present a strong case for vital reform in test use, if not for their total abandonment in research.The book is designed for practicing researchers-those not extensively trained in mathematics and statistics that must nevertheless regularly decide if and how tests of significance are to be used-and for those training for research. While controversy has been centered in sociology and psychology, and the book will be especially useful to researchers and students in those fields, its importance is great across the spectrum of the scientific disciplines in which statistical procedures are essential-notably political science, economics, and the other social sciences, education, and many biological fields as well.Denton E. Morrison is professor, Department of Sociology, Michigan State University.Ramon E. Henkel is associate professor emeritus, Department of Sociology University of Maryland. He teaches as part of the graduate faculty.

This book explains the misuses and abuses of Null Hypothesis Significance Tests, which are reconsidered in light of Jeffreys’ Bayesian concept of the role of statistical inference, in experimental investigations. Minimizing the technical aspects, the studies focuses mainly on methodological contributions. The first part of the book gives an overview of the major approaches to statistical testing and an enlightening discussion of the philosophies of Fisher, Neyman-Pearson and Jeffrey. The conceptual and methodological implications of current practices of reporting effect sizes and confidence intervals are also examined and challenged. This sheds new light on the "significance testing controversy" and provides an appropriate Bayesian framework for a comprehensive approach to the analysis and interpretation of experimental data. The second part of the book provides concrete Bayesian routine procedures that bypass common misuses of significance testing and are readily applicable in a wide range of real applications. This approach addresses the need for objective reporting of experimental data, that is acceptable to the scientific community. This is emphasized by the name fiducial (from the Latin fiducia = confidence). The fiducial Bayesian procedures provide the reader with a real opportunity to think sensibly about problems of statistical inference. This book prepares students and researchers to critically read statistical analyses reported in the literature and equips them with an appropriate alternative to the use of significance testing.

Traditional education in statistics that emphasises significance testing leaves researchers and students ill prepared to understand what their results really mean. Specifically, most researchers and students who do not have strong quantitative backgrounds have difficulty understanding outcomes of statistical tests. As more and more people become aware of this problem, the emphasis on statistical significance in the reporting of results is declining. Increasingly, researchers are expected to describe the magnitudes and precisions of their findings and also their practical, theoretical, or clinical significance. This accessibly written book reviews the controversy about significance testing, which has now crossed various disciplines as diverse as psychology, ecology, commerce, education, and biology, among others. It also introduces readers to alternative methods, especially effect size estimation (at both the group and case levels) and interval estimation (confidence intervals) in comparative studies. Basics of bootstrapping and Bayesian estimation are also considered. Research examples from substance abuse, education, learning, and other areas illustrate how to apply these methods. A companion website promotes learning by providing chapter exercises and sample answers, downloadable raw data files for many research examples, and links to other useful websites. New to this edition is coverage of robust statistical methods for parameter estimation, effect size estimation, and interval estimation. A new chapter covers the logic and illogic of significance testing. This edition also addresses recent developments such as the new requirements of some journals for the reporting of effect sizes.

The classic edition of What If There Were No Significance Tests? highlights current statistical inference practices. Four areas are featured as essential for making inferences: sound judgment, meaningful research questions, relevant design, and assessing fit in multiple ways. Other options (data visualization, replication or meta-analysis), other features (mediation, moderation, multiple levels or classes), and other approaches (Bayesian analysis, simulation, data mining, qualitative inquiry) are also suggested. The Classic Edition’s new Introduction demonstrates the ongoing relevance of the topic and the charge to move away from an exclusive focus on NHST, along with new methods to help make significance testing more accessible to a wider body of researchers to improve our ability to make more accurate statistical inferences. Part 1 presents an overview of significance testing issues. The next part discusses the debate in which significance testing should be rejected or retained. The third part outlines various methods that may supplement significance testing procedures. Part 4 discusses Bayesian approaches and methods and the use of confidence intervals versus significance tests. The book concludes with philosophy of science perspectives. Rather than providing definitive prescriptions, the chapters are largely suggestive of general issues, concerns, and application guidelines. The editors allow readers to choose the best way to conduct hypothesis testing in their respective fields. For anyone doing research in the social sciences, this book is bound to become "must" reading. Ideal for use as a supplement for graduate courses in statistics or quantitative analysis taught in psychology, education, business, nursing, medicine, and the social sciences, the book also benefits independent researchers in the behavioral and social sciences and those who teach statistics.

In this book, an integrated introduction to statistical inference is provided from a frequentist likelihood-based viewpoint. Classical results are presented together with recent developments, largely built upon ideas due to R.A. Fisher. The term ?neo-Fisherian? highlights this.After a unified review of background material (statistical models, likelihood, data and model reduction, first-order asymptotics) and inference in the presence of nuisance parameters (including pseudo-likelihoods), a self-contained introduction is given to exponential families, exponential dispersion models, generalized linear models, and group families. Finally, basic results of higher-order asymptotics are introduced (index notation, asymptotic expansions for statistics and distributions, and major applications to likelihood inference).The emphasis is more on general concepts and methods than on regularity conditions. Many examples are given for specific statistical models. Each chapter is supplemented with problems and bibliographic notes. This volume can serve as a textbook in intermediate-level undergraduate and postgraduate courses in statistical inference.

Examining the major approaches to hypothesis testing and model selection, this book blends statistical theory with recommendations for practice, illustrated with real-world social science examples. It systematically compares classical (frequentist) and Bayesian approaches, showing how they are applied, exploring ways to reconcile the differences between them, and evaluating key controversies and criticisms. The book also addresses the role of hypothesis testing in the evaluation of theories, the relationship between hypothesis tests and confidence intervals, and the role of prior knowledge in Bayesian estimation and Bayesian hypothesis testing. Two easily calculated alternatives to standard hypothesis tests are discussed in depth: the Akaike information criterion (AIC) and Bayesian information criterion (BIC). The companion website ([ital]www.guilford.com/weakliem-materials[/ital]) supplies data and syntax files for the book's examples.