Implementing Randomized Field Trials In Education

The central idea of evidence-based education-that education policy and practice ought to be fashioned based on what is known from rigorous research-offers a compelling way to approach reform efforts. Recent federal trends reflect a growing enthusiasm for such change. Most visibly, the 2002 No Child Left Behind Act requires that "scientifically based [education] research" drive the use of federal education funds at the state and local levels. This emphasis is also reflected in a number of government and nongovernment initiatives across the country. As consensus builds around the goals of evidence-based education, consideration of what it will take to make it a reality becomes the crucial next step. In this context, the Center for Education of the National Research Council (NRC) has undertaken a series of activities to address issues related to the quality of scientific education research. In 2002, the NRC released Scientific Research in Education (National Research Council, 2002), a report designed to articulate the nature of scientific education research and to guide efforts aimed at improving its quality. Building on this work, the Committee on Research in Education was convened to advance an improved understanding of a scientific approach to addressing education problems; to engage the field of education research in action-oriented dialogue about how to further the accumulation of scientific knowledge; and to coordinate, support, and promote cross-fertilization among NRC efforts in education research. The main locus of activity undertaken to meet these objectives was a year-long series of workshops. This report is a summary of the third workshop in the series, on the implementation and implications of randomized field trials in education.

Abstract curricular program implementation in the context of randomized field trials Gloria Isabel Miller This study examined three cases of commercially available curricular program implementations to determine if a unified approach to measuring the level of implementation was possible (proof of concept). Further, the study investigated whether the level of curriculum and implementation plan specificity made a difference to the strength of implementation achieved in classrooms; and described the implementation evolution in different contexts. The study sample consists of a total of 163 teachers in eight school districts across the United States. In each case teachers were randomly assigned to using the curricular innovation or their currently used materials and processes. The three cases, HS-Math, NewScience, and MathIntervention, were purposely chosen to represent three different points of curricular and implementation specificity and two different subject areas, math and science. Each case features a commercially available program that also had opportunities for teachers to use "electronic" technology to enhance their learning or to engage their students. The cases represent differing student grade levels. The cases are different enough to provide a range that exercises the measurement techniques introduced in this study so results can begin to generalize across curricular programs and grades. However, the cases are similar enough in research design, instrumentation, and data collection methods to make them comparable. A key contribution of this investigation is the creation of a framework to measure the level of implementation (the extent to which the teacher and students display the actions, behaviors, and interactions expected by using the innovation). The unified conceptual framework arrived at by using an Activity Theory perspective together with the analytical methods employed provide a way to view the rich complex interaction of implementation as a system with the larger system of the school organization. Data from the analysis revealed that variations in the level of implementation were no different regardless of the level of specificity. A strong finding of this work is that implementation evolves slowly even when the curricular program is scripted and coaching support is provided to teachers. The paper concludes with implications for policy and future research.

Opinions about education programs and practices are offered frequently—by children, parents, teachers, and policymakers. Credible studies of the impact of programs on the performance of children are far less frequent. Researchers use a variety of tools to determine their impact and efficacy, including sample surveys, narrative studies, and exploratory research. However, randomized field trials, which are commonly used in other disciplines, are rarely employed to measure the impact of education practice. Evidence Matters explores the history and current status of research in education and encourages the more frequent use of such trials. Judith Gueron (Manpower Demonstration Research Corporation), discusses the challenges involved in randomized trials and offers practical advice drawn experience. Robert Boruch (Wharton School, University of Pennsylvania), Dorothy de Moya (Campbell Collaboration Secretariat), and Brooke Snyder (University of Pennsylvania) explore the use of randomized field trials in education and other fields. David Cohen, Stephen Raudenbush, and Deborah Loewenberg Ball (all from the University of Michigan) review the history of progress in education over the past forty years and urge increased research on coherent instruction regimes. Maris Vinovskis (University of Michigan) examines the history and role of the U.S. Department of Education in developing rigorous evaluation of federal programs, and suggests a new National Center for Evaluation and Development. Thomas Cook and Monique Renee Payne (both from Northwestern University) take on the claim that randomized field trials are inappropriate in the U.S. education system. Gary Burtless (Brookings Institution) explores the political and professional factors that influence randomized field trials in economic programs, examining possible explanations for their lack of frequent use in education. Carol Weiss (Harvard University) provides a brief history of community studies in the United States and suggests a variety of alternatives to randomization. It is difficult to gauge the impact of various approaches in education. But the authors give a variety of concrete examples to illustrate the feasibility of randomized trials, and the circumstances under which they are appropriate. By offering a variety of suggestions to improve the methods used to evaluate education programs, the contributors to this volume seek to improve education in the United States. Frederick Mosteller is Roger I. Lee Professor in Mathematical Statistics, emeritus, in the department of statistics at Harvard University. Robert Boruch is the University Trustee Chair Professor the graduate school of education and statistics department at the Wharton School, University of Pennsylvania

Argues against the “culture of science” currently dominating education discourse and in favor of a more critical understanding of various modes of inquiry.

Transforming education into an evidence-based field depends in no small part on a strong base of scientific knowledge to inform educational policy and practice. Advancing Scientific Research in Education makes select recommendations for strengthening scientific education research and targets federal agencies, professional associations, and universitiesâ€"particularly schools of educationâ€"to take the lead in advancing the field.

The field of education has experienced extraordinary technological, societal, and institutional change in recent years, making it one of the most fascinating yet complex fields of study in social science. Unequalled in its combination of authoritative scholarship and comprehensive coverage, International Encyclopedia of Education, Third Edition succeeds two highly successful previous editions (1985, 1994) in aiming to encapsulate research in this vibrant field for the twenty-first century reader. Under development for five years, this work encompasses over 1,000 articles across 24 individual areas of coverage, and is expected to become the dominant resource in the field. Education is a multidisciplinary and international field drawing on a wide range of social sciences and humanities disciplines, and this new edition comprehensively matches this diversity. The diverse background and multidisciplinary subject coverage of the Editorial Board ensure a balanced and objective academic framework, with 1,500 contributors representing over 100 countries, capturing a complete portrait of this evolving field. A totally new work, revamped with a wholly new editorial board, structure and brand-new list of meta-sections and articles Developed by an international panel of editors and authors drawn from senior academia Web-enhanced with supplementary multimedia audio and video files, hotlinked to relevant references and sources for further study Incorporates ca. 1,350 articles, with timely coverage of such topics as technology and learning, demography and social change, globalization, and adult learning, to name a few Offers two content delivery options - print and online - the latter of which provides anytime, anywhere access for multiple users and superior search functionality via ScienceDirect, as well as multimedia content, including audio and video files

Co-published by Routledge for the American Educational Research Association (AERA) Educational policy continues to be of major concern. Policy debates about economic growth and national competitiveness, for example, commonly focus on the importance of human capital and a highly educated workforce. Defining the theoretical boundaries and methodological approaches of education policy research are the two primary themes of this comprehensive, AERA-sponsored Handbook. Organized into seven sections, the Handbook focuses on (1) disciplinary foundations of educational policy, (2) methodological perspectives, (3) the policy process, (4) resources, management, and organization, (5) teaching and learning policy, (6) actors and institutions, and (7) education access and differentiation. Drawing from multiple disciplines, the Handbook’s over one hundred authors address three central questions: What policy issues and questions have oriented current policy research? What research strategies and methods have proven most fruitful? And what issues, questions, and methods will drive future policy research? Topics such as early childhood education, school choice, access to higher education, teacher accountability, and testing and measurement cut across the 63 chapters in the volume. The politics surrounding these and other issues are objectively analyzed by authors and commentators. Each of the seven sections concludes with two commentaries by leading scholars in the field. The first considers the current state of policy design, and the second addresses the current state of policy research. This book is appropriate for scholars and graduate students working in the field of education policy and for the growing number of academic, government, and think-tank researchers engaged in policy research. For more information on the American Educational Research Association, please visit: http://www.aera.net/.