Implementing Randomized Field Trials In Education

The central idea of evidence-based education-that education policy and practice ought to be fashioned based on what is known from rigorous research-offers a compelling way to approach reform efforts. Recent federal trends reflect a growing enthusiasm for such change. Most visibly, the 2002 No Child Left Behind Act requires that "scientifically based [education] research" drive the use of federal education funds at the state and local levels. This emphasis is also reflected in a number of government and nongovernment initiatives across the country. As consensus builds around the goals of evidence-based education, consideration of what it will take to make it a reality becomes the crucial next step. In this context, the Center for Education of the National Research Council (NRC) has undertaken a series of activities to address issues related to the quality of scientific education research. In 2002, the NRC released Scientific Research in Education (National Research Council, 2002), a report designed to articulate the nature of scientific education research and to guide efforts aimed at improving its quality. Building on this work, the Committee on Research in Education was convened to advance an improved understanding of a scientific approach to addressing education problems; to engage the field of education research in action-oriented dialogue about how to further the accumulation of scientific knowledge; and to coordinate, support, and promote cross-fertilization among NRC efforts in education research. The main locus of activity undertaken to meet these objectives was a year-long series of workshops. This report is a summary of the third workshop in the series, on the implementation and implications of randomized field trials in education.

Opinions about education programs and practices are offered frequently—by children, parents, teachers, and policymakers. Credible studies of the impact of programs on the performance of children are far less frequent. Researchers use a variety of tools to determine their impact and efficacy, including sample surveys, narrative studies, and exploratory research. However, randomized field trials, which are commonly used in other disciplines, are rarely employed to measure the impact of education practice. Evidence Matters explores the history and current status of research in education and encourages the more frequent use of such trials. Judith Gueron (Manpower Demonstration Research Corporation), discusses the challenges involved in randomized trials and offers practical advice drawn experience. Robert Boruch (Wharton School, University of Pennsylvania), Dorothy de Moya (Campbell Collaboration Secretariat), and Brooke Snyder (University of Pennsylvania) explore the use of randomized field trials in education and other fields. David Cohen, Stephen Raudenbush, and Deborah Loewenberg Ball (all from the University of Michigan) review the history of progress in education over the past forty years and urge increased research on coherent instruction regimes. Maris Vinovskis (University of Michigan) examines the history and role of the U.S. Department of Education in developing rigorous evaluation of federal programs, and suggests a new National Center for Evaluation and Development. Thomas Cook and Monique Renee Payne (both from Northwestern University) take on the claim that randomized field trials are inappropriate in the U.S. education system. Gary Burtless (Brookings Institution) explores the political and professional factors that influence randomized field trials in economic programs, examining possible explanations for their lack of frequent use in education. Carol Weiss (Harvard University) provides a brief history of community studies in the United States and suggests a variety of alternatives to randomization. It is difficult to gauge the impact of various approaches in education. But the authors give a variety of concrete examples to illustrate the feasibility of randomized trials, and the circumstances under which they are appropriate. By offering a variety of suggestions to improve the methods used to evaluate education programs, the contributors to this volume seek to improve education in the United States. Frederick Mosteller is Roger I. Lee Professor in Mathematical Statistics, emeritus, in the department of statistics at Harvard University. Robert Boruch is the University Trustee Chair Professor the graduate school of education and statistics department at the Wharton School, University of Pennsylvania

Abstract curricular program implementation in the context of randomized field trials Gloria Isabel Miller This study examined three cases of commercially available curricular program implementations to determine if a unified approach to measuring the level of implementation was possible (proof of concept). Further, the study investigated whether the level of curriculum and implementation plan specificity made a difference to the strength of implementation achieved in classrooms; and described the implementation evolution in different contexts. The study sample consists of a total of 163 teachers in eight school districts across the United States. In each case teachers were randomly assigned to using the curricular innovation or their currently used materials and processes. The three cases, HS-Math, NewScience, and MathIntervention, were purposely chosen to represent three different points of curricular and implementation specificity and two different subject areas, math and science. Each case features a commercially available program that also had opportunities for teachers to use "electronic" technology to enhance their learning or to engage their students. The cases represent differing student grade levels. The cases are different enough to provide a range that exercises the measurement techniques introduced in this study so results can begin to generalize across curricular programs and grades. However, the cases are similar enough in research design, instrumentation, and data collection methods to make them comparable. A key contribution of this investigation is the creation of a framework to measure the level of implementation (the extent to which the teacher and students display the actions, behaviors, and interactions expected by using the innovation). The unified conceptual framework arrived at by using an Activity Theory perspective together with the analytical methods employed provide a way to view the rich complex interaction of implementation as a system with the larger system of the school organization. Data from the analysis revealed that variations in the level of implementation were no different regardless of the level of specificity. A strong finding of this work is that implementation evolves slowly even when the curricular program is scripted and coaching support is provided to teachers. The paper concludes with implications for policy and future research.

The use of randomised controlled trials (RCTs), most commonly a medical sciences research tool, is a hotly debated topic in Education. This book examines the controversial aspects of RCTs in Education and sets out the potential and pitfalls of the method. Drawing on their own extensive experience of running RCTs, and their work at the Centre for Evidence and Social Innovation (CESI) at Queen’s University, Belfast, the authors provide a thorough practical introduction to the use of randomised controlled trials in education. Using real data sets, chapters equip the reader with all of the key knowledge and skills required to design, run, analyse and report an RCT. Coverage includes: · Step-by-step guidance on analysing data · How to assess the reliability and validity of results · Advice on balancing the demands of various stakeholders Essential reading for postgraduate and more experienced researchers, as well as teachers and educationalists seeking to increase their knowledge and understanding of the use of such methods in education.

Argues against the “culture of science” currently dominating education discourse and in favor of a more critical understanding of various modes of inquiry.

Transforming education into an evidence-based field depends in no small part on a strong base of scientific knowledge to inform educational policy and practice. Advancing Scientific Research in Education makes select recommendations for strengthening scientific education research and targets federal agencies, professional associations, and universitiesâ€"particularly schools of educationâ€"to take the lead in advancing the field.

The field of education has experienced extraordinary technological, societal, and institutional change in recent years, making it one of the most fascinating yet complex fields of study in social science. Unequalled in its combination of authoritative scholarship and comprehensive coverage, International Encyclopedia of Education, Third Edition succeeds two highly successful previous editions (1985, 1994) in aiming to encapsulate research in this vibrant field for the twenty-first century reader. Under development for five years, this work encompasses over 1,000 articles across 24 individual areas of coverage, and is expected to become the dominant resource in the field. Education is a multidisciplinary and international field drawing on a wide range of social sciences and humanities disciplines, and this new edition comprehensively matches this diversity. The diverse background and multidisciplinary subject coverage of the Editorial Board ensure a balanced and objective academic framework, with 1,500 contributors representing over 100 countries, capturing a complete portrait of this evolving field. A totally new work, revamped with a wholly new editorial board, structure and brand-new list of meta-sections and articles Developed by an international panel of editors and authors drawn from senior academia Web-enhanced with supplementary multimedia audio and video files, hotlinked to relevant references and sources for further study Incorporates ca. 1,350 articles, with timely coverage of such topics as technology and learning, demography and social change, globalization, and adult learning, to name a few Offers two content delivery options - print and online - the latter of which provides anytime, anywhere access for multiple users and superior search functionality via ScienceDirect, as well as multimedia content, including audio and video files