The Future Of Meta Analysis

Scientific progress often begins with the difficult task of preparing informed, conclusive reviews of existing research. Since the 1970s, the traditional "subjective" approach to research reviewing in the social sciences has been challenged by a statistical alternative known as meta-analysis. Meta-analysis provides a principled method of distilling reliable generalizations from previous studies on a single topic, thereby providing a quantitative and objective background for future research. The Future of Meta-Analysis brings together expert researchers for an in-depth examination of this new methodology—not to promote a consensus view but rather to explore from several perspectives the theories, tensions, and concerns of meta-analysis, and to illustrate through concrete examples the rationale behind meta-analytic decisions. In a meta-analysis prepared especially for this volume, a statistician and a psychologist review the existing literature on aphasia treatment. In a second study, experts analyze six still-unpublished meta-analyses sponsored by the National Institute of Education to investigate the effects of school desegregation on the academic achievement of black children. This unique case study approach provides valuable discussion of the process of meta-analysis and of the current implications of meta-analysis for policy assessment. Prepared under the auspices of the National Research Council, The Future of Meta-Analysis presents a forum for leaders in this rapidly evolving field to discuss salient conceptual and technical issues and to offer a new theoretical framework, further methodological guidance, and statistical innovations that anticipate a future in which meta-analysis will play an even more effective and valuable role in social science research.

Scientific progress often begins with the difficult task of preparing informed, conclusive reviews of existing research. Since the 1970s, the traditional "subjective" approach to research reviewing in the social sciences has been challenged by a statistical alternative known as meta-analysis. Meta-analysis provides a principled method of distilling reliable generalizations from previous studies on a single topic, thereby providing a quantitative and objective background for future research. The Future of Meta-Analysis brings together expert researchers for an in-depth examination of this new methodology—not to promote a consensus view but rather to explore from several perspectives the theories, tensions, and concerns of meta-analysis, and to illustrate through concrete examples the rationale behind meta-analytic decisions. In a meta-analysis prepared especially for this volume, a statistician and a psychologist review the existing literature on aphasia treatment. In a second study, experts analyze six still-unpublished meta-analyses sponsored by the National Institute of Education to investigate the effects of school desegregation on the academic achievement of black children. This unique case study approach provides valuable discussion of the process of meta-analysis and of the current implications of meta-analysis for policy assessment. Prepared under the auspices of the National Research Council, The Future of Meta-Analysis presents a forum for leaders in this rapidly evolving field to discuss salient conceptual and technical issues and to offer a new theoretical framework, further methodological guidance, and statistical innovations that anticipate a future in which meta-analysis will play an even more effective and valuable role in social science research.

Healthcare providers, consumers, researchers and policy makers are inundated with unmanageable amounts of information, including evidence from healthcare research. It has become impossible for all to have the time and resources to find, appraise and interpret this evidence and incorporate it into healthcare decisions. Cochrane Reviews respond to this challenge by identifying, appraising and synthesizing research-based evidence and presenting it in a standardized format, published in The Cochrane Library (www.thecochranelibrary.com). The Cochrane Handbook for Systematic Reviews of Interventions contains methodological guidance for the preparation and maintenance of Cochrane intervention reviews. Written in a clear and accessible format, it is the essential manual for all those preparing, maintaining and reading Cochrane reviews. Many of the principles and methods described here are appropriate for systematic reviews applied to other types of research and to systematic reviews of interventions undertaken by others. It is hoped therefore that this book will be invaluable to all those who want to understand the role of systematic reviews, critically appraise published reviews or perform reviews themselves.

Individual Participant Data Meta-Analysis: A Handbook for Healthcare Research provides a comprehensive introduction to the fundamental principles and methods that healthcare researchers need when considering, conducting or using individual participant data (IPD) meta-analysis projects. Written and edited by researchers with substantial experience in the field, the book details key concepts and practical guidance for each stage of an IPD meta-analysis project, alongside illustrated examples and summary learning points. Split into five parts, the book chapters take the reader through the journey from initiating and planning IPD projects to obtaining, checking, and meta-analysing IPD, and appraising and reporting findings. The book initially focuses on the synthesis of IPD from randomised trials to evaluate treatment effects, including the evaluation of participant-level effect modifiers (treatment-covariate interactions). Detailed extension is then made to specialist topics such as diagnostic test accuracy, prognostic factors, risk prediction models, and advanced statistical topics such as multivariate and network meta-analysis, power calculations, and missing data. Intended for a broad audience, the book will enable the reader to: Understand the advantages of the IPD approach and decide when it is needed over a conventional systematic review Recognise the scope, resources and challenges of IPD meta-analysis projects Appreciate the importance of a multi-disciplinary project team and close collaboration with the original study investigators Understand how to obtain, check, manage and harmonise IPD from multiple studies Examine risk of bias (quality) of IPD and minimise potential biases throughout the project Understand fundamental statistical methods for IPD meta-analysis, including two-stage and one-stage approaches (and their differences), and statistical software to implement them Clearly report and disseminate IPD meta-analyses to inform policy, practice and future research Critically appraise existing IPD meta-analysis projects Address specialist topics such as effect modification, multiple correlated outcomes, multiple treatment comparisons, non-linear relationships, test accuracy at multiple thresholds, multiple imputation, and developing and validating clinical prediction models Detailed examples and case studies are provided throughout.

Doing Meta-Analysis with R: A Hands-On Guide serves as an accessible introduction on how meta-analyses can be conducted in R. Essential steps for meta-analysis are covered, including calculation and pooling of outcome measures, forest plots, heterogeneity diagnostics, subgroup analyses, meta-regression, methods to control for publication bias, risk of bias assessments and plotting tools. Advanced but highly relevant topics such as network meta-analysis, multi-three-level meta-analyses, Bayesian meta-analysis approaches and SEM meta-analysis are also covered. A companion R package, dmetar, is introduced at the beginning of the guide. It contains data sets and several helper functions for the meta and metafor package used in the guide. The programming and statistical background covered in the book are kept at a non-expert level, making the book widely accessible. Features • Contains two introductory chapters on how to set up an R environment and do basic imports/manipulations of meta-analysis data, including exercises • Describes statistical concepts clearly and concisely before applying them in R • Includes step-by-step guidance through the coding required to perform meta-analyses, and a companion R package for the book

Policymakers, medical practitioners, and the public alike face an increasingly bewildering flood of new and often contradictory scientific studies on almost every topic. Whether the issue is the the best treatment for breast cancer, the need for prenatal food programs to improve the health of poor infants and mothers, or the ability of women to succeed in scientific professions, the healthy growth of modern science has at times done more to stir up controversy than to establish reliable knowledge. But now scientists in several fields have developed a sophisticated new methodology called meta-analysis to address this problem. By numerically combining diverse research findings on a single question, meta-analysis can be used to identify their central tendency and reach conclusions far more reliable than those of any single investigation. How Science Takes Stock vividly tells the story of meta-analysis through the eyes of its architects and champions, and chronicles its history, techniques, achievements, and controversies. Noted science author Morton Hunt visits key practitioners and recounts their use of meta-analysis to resolve important scientific puzzles and longstanding debates. Does psychotherapy work, and if so what form works best? Does spending federal money on education really improve student performance? Can a single enzyme significantly decrease the risk of heart attack? Do boot camps reduce juvenile delinquency? With each account, Hunt illustrates the major components of the meta-analytic method, reveals strategies for resolving practical and theoretical problems, and discusses the impact of meta-analysis on the science and policy communities. In many cases, he demonstrates how meta-analysts have gone a step further to determine the causes of earlier discrepancies. In this way they not only identify successful approaches to the question at hand, but also clarify the conditions under which they will work best. Hunt also portrays the important but frequently controversial business of doing meta-analysis for legislators and government agencies, particularly in sensitive areas of social policy. How Science Takes Stock demonstrates how the statistical techniques of meta-analysis produce more accurate data than the standard literature review or the old-fashioned process of tallying up the results of each scientific study as if they were votes in an election to decide the truth. Hunt also addresses issues of quality control in each phase of the meta-analytic process, and answers skeptics who claim that the dissimilarities between studies are often too significant for meta-analysis to be any more than an apples and oranges approach. This volume conveys the power of meta-analysis to help social policymakers and health professionals resolve their most pressing problems. How Science Takes Stock concludes with a discussion of the future of meta-analysis that examines its potential for further refinements, its growth in the scientific literature, and exciting new possibilities for its future use. An appendix by meta-analysis expert Harris Cooper offers some finer points on the mechanics of conducting a meta-analytic investigation.

Research synthesis is the practice of systematically distilling and integrating data from many studies in order to draw more reliable conclusions about a given research issue. When the first edition of The Handbook of Research Synthesis and Meta-Analysis was published in 1994, it quickly became the definitive reference for conducting meta-analyses in both the social and behavioral sciences. In the third edition, editors Harris Cooper, Larry Hedges, and Jeff Valentine present updated versions of classic chapters and add new sections that evaluate cutting-edge developments in the field. The Handbook of Research Synthesis and Meta-Analysis draws upon groundbreaking advances that have transformed research synthesis from a narrative craft into an important scientific process in its own right. The editors and leading scholars guide the reader through every stage of the research synthesis process—problem formulation, literature search and evaluation, statistical integration, and report preparation. The Handbook incorporates state-of-the-art techniques from all quantitative synthesis traditions and distills a vast literature to explain the most effective solutions to the problems of quantitative data integration. Among the statistical issues addressed are the synthesis of non-independent data sets, fixed and random effects methods, the performance of sensitivity analyses and model assessments, the development of machine-based abstract screening, the increased use of meta-regression and the problems of missing data. The Handbook also addresses the non-statistical aspects of research synthesis, including searching the literature and developing schemes for gathering information from study reports. Those engaged in research synthesis will find useful advice on how tables, graphs, and narration can foster communication of the results of research syntheses. The third edition of the Handbook provides comprehensive instruction in the skills necessary to conduct research syntheses and represents the premier text on research synthesis. Praise for the first edition: "The Handbook is a comprehensive treatment of literature synthesis and provides practical advice for anyone deep in the throes of, just teetering on the brink of, or attempting to decipher a meta-analysis. Given the expanding application and importance of literature synthesis, understanding both its strengths and weaknesses is essential for its practitioners and consumers. This volume is a good beginning for those who wish to gain that understanding." —Chance "Meta-analysis, as the statistical analysis of a large collection of results from individual studies is called, has now achieved a status of respectability in medicine. This respectability, when combined with the slight hint of mystique that sometimes surrounds meta-analysis, ensures that results of studies that use it are treated with the respect they deserve....The Handbook of Research Synthesis is one of the most important publications in this subject both as a definitive reference book and a practical manual."—British Medical Journal When the first edition of The Handbook of Research Synthesis was published in 1994, it quickly became the definitive reference for researchers conducting meta-analyses of existing research in both the social and biological sciences. In this fully revised second edition, editors Harris Cooper, Larry Hedges, and Jeff Valentine present updated versions of the Handbook's classic chapters, as well as entirely new sections reporting on the most recent, cutting-edge developments in the field. Research synthesis is the practice of systematically distilling and integrating data from a variety of sources in order to draw more reliable conclusions about a given question or topic. The Handbook of Research Synthesis and Meta-Analysis draws upon years of groundbreaking advances that have transformed research synthesis from a narrative craft into an important scientific process in its own right. Cooper, Hedges, and Valentine have assembled leading authorities in the field to guide the reader through every stage of the research synthesis process—problem formulation, literature search and evaluation, statistical integration, and report preparation. The Handbook of Research Synthesis and Meta-Analysis incorporates state-of-the-art techniques from all quantitative synthesis traditions. Distilling a vast technical literature and many informal sources, the Handbook provides a portfolio of the most effective solutions to the problems of quantitative data integration. Among the statistical issues addressed by the authors are the synthesis of non-independent data sets, fixed and random effects methods, the performance of sensitivity analyses and model assessments, and the problem of missing data. The Handbook of Research Synthesis and Meta-Analysis also provides a rich treatment of the non-statistical aspects of research synthesis. Topics include searching the literature, and developing schemes for gathering information from study reports. Those engaged in research synthesis will also find useful advice on how tables, graphs, and narration can be used to provide the most meaningful communication of the results of research synthesis. In addition, the editors address the potentials and limitations of research synthesis, and its future directions. The past decade has been a period of enormous growth in the field of research synthesis. The second edition Handbook thoroughly revises original chapters to assure that the volume remains the most authoritative source of information for researchers undertaking meta-analysis today. In response to the increasing use of research synthesis in the formation of public policy, the second edition includes a new chapter on both the strengths and limitations of research synthesis in policy debates

This is the first introductory statistics text to use an estimation approach from the start to help readers understand effect sizes, confidence intervals (CIs), and meta-analysis (‘the new statistics’). It is also the first text to explain the new and exciting Open Science practices, which encourage replication and enhance the trustworthiness of research. In addition, the book explains NHST fully so students can understand published research. Numerous real research examples are used throughout. The book uses today’s most effective learning strategies and promotes critical thinking, comprehension, and retention, to deepen users’ understanding of statistics and modern research methods. The free ESCI (Exploratory Software for Confidence Intervals) software makes concepts visually vivid, and provides calculation and graphing facilities. The book can be used with or without ESCI. Other highlights include: - Coverage of both estimation and NHST approaches, and how to easily translate between the two. - Some exercises use ESCI to analyze data and create graphs including CIs, for best understanding of estimation methods. -Videos of the authors describing key concepts and demonstrating use of ESCI provide an engaging learning tool for traditional or flipped classrooms. -In-chapter exercises and quizzes with related commentary allow students to learn by doing, and to monitor their progress. -End-of-chapter exercises and commentary, many using real data, give practice for using the new statistics to analyze data, as well as for applying research judgment in realistic contexts. -Don’t fool yourself tips help students avoid common errors. -Red Flags highlight the meaning of "significance" and what p values actually mean. -Chapter outlines, defined key terms, sidebars of key points, and summarized take-home messages provide a study tool at exam time. -http://www.routledge.com/cw/cumming offers for students: ESCI downloads; data sets; key term flashcards; tips for using SPSS for analyzing data; and videos. For instructors it offers: tips for teaching the new statistics and Open Science; additional homework exercises; assessment items; answer keys for homework and assessment items; and downloadable text images; and PowerPoint lecture slides. Intended for introduction to statistics, data analysis, or quantitative methods courses in psychology, education, and other social and health sciences, researchers interested in understanding the new statistics will also appreciate this book. No familiarity with introductory statistics is assumed.