Automated Scoring Of Complex Tasks In Computer Based Testing

This is the first volume to provide the latest methods and examples of "best practices" in the design, implementation, and evaluation of automated scoring for complex assessments. The contributing authors, all noted leaders in the field, introduce each m

Automated Speaking Assessment: Using Language Technologies to Score Spontaneous Speech provides a thorough overview of state-of-the-art automated speech scoring technology as it is currently used at Educational Testing Service (ETS). Its main focus is related to the automated scoring of spontaneous speech elicited by TOEFL iBT Speaking section items, but other applications of speech scoring, such as for more predictable spoken responses or responses provided in a dialogic setting, are also discussed. The book begins with an in-depth overview of the nascent field of automated speech scoring—its history, applications, and challenges—followed by a discussion of psychometric considerations for automated speech scoring. The second and third parts discuss the integral main components of an automated speech scoring system as well as the different types of automatically generated measures extracted by the system features related to evaluate the speaking construct of communicative competence as measured defined by the TOEFL iBT Speaking assessment. Finally, the last part of the book touches on more recent developments, such as providing more detailed feedback on test takers’ spoken responses using speech features and scoring of dialogic speech. It concludes with a discussion, summary, and outlook on future developments in this area. Written with minimal technical details for the benefit of non-experts, this book is an ideal resource for graduate students in courses on Language Testing and Assessment as well as teachers and researchers in applied linguistics.

The arrival of the computer in educational and psychological testing has led to the current popularity of adaptive testing---a testing format in which the computer uses statistical information about the test items to automatically adapt their selection to a real-time update of the test taker’s ability estimate. This book covers such key features of adaptive testing as item selection and ability estimation, adaptive testing with multidimensional abilities, sequencing adaptive test batteries, multistage adaptive testing, item-pool design and maintenance, estimation of item and item-family parameters, item and person fit, as well as adaptive mastery and classification testing. It also shows how these features are used in the daily operations of several large-scale adaptive testing programs.

First Published in 2008. Routledge is an imprint of Taylor & Francis, an informa company.

From early answer sheets filled in with number 2 pencils, to tests administered by mainframe computers, to assessments wholly constructed by computers, it is clear that technology is changing the field of educational and psychological measurement. The numerous and rapid advances have immediate impact on test creators, assessment professionals, and those who implement and analyze assessments. This comprehensive new volume brings together leading experts on the issues posed by technological applications in testing, with chapters on game-based assessment, testing with simulations, video assessment, computerized test development, large-scale test delivery, model choice, validity, and error issues. Including an overview of existing literature and ground-breaking research, each chapter considers the technological, practical, and ethical considerations of this rapidly-changing area. Ideal for researchers and professionals in testing and assessment, Technology and Testing provides a critical and in-depth look at one of the most pressing topics in educational testing today.

"Automated scoring engines [...] require a careful balancing of the contributions of technology, NLP, psychometrics, artificial intelligence, and the learning sciences. The present handbook is evidence that the theories, methodologies, and underlying technology that surround automated scoring have reached maturity, and that there is a growing acceptance of these technologies among experts and the public." From the Foreword by Alina von Davier, ACTNext Senior Vice President Handbook of Automated Scoring: Theory into Practice provides a scientifically grounded overview of the key research efforts required to move automated scoring systems into operational practice. It examines the field of automated scoring from the viewpoint of related scientific fields serving as its foundation, the latest developments of computational methodologies utilized in automated scoring, and several large-scale real-world applications of automated scoring for complex learning and assessment systems. The book is organized into three parts that cover (1) theoretical foundations, (2) operational methodologies, and (3) practical illustrations, each with a commentary. In addition, the handbook includes an introduction and synthesis chapter as well as a cross-chapter glossary.

The increasingly complex environment of the 21st century demands unprecedented knowledge, skills and abilities for people from all walks of life. One powerful solution that blends the science of learning with the technological advances of computing is Virtual Environments. In the United States alone, the Department of Defense has invested billions of dollars over the past decade to make this field and its developments as effective as possible. This 3-volume work provides, for the first time, comprehensive coverage of the many different domains that must be integrated for Virtual Environments to fully provide effective training and education. The first volume is dedicated to a thorough understanding of learning theory, requirements definition and performance measurement, providing insight into the human-centric specifications the VE must satisfy to succeed. Volume II provides the latest information on VE component technologies, and Volume III offers discussion of an extensive collection of integrated systems presented as VE use-cases, and results of effectiveness evaluation studies. The text includes emerging directions of this evolving technology, from cognitive rehabilitation to the next generation of museum exhibitions. Finally, the handbook offers a glimpse into the future with this fascinating technology. This groundbreaking set will interest students, scholars and researchers in the fields of military science, technology, computer science, business, law enforcement, cognitive psychology, education and health. Topics addressed include guidance and interventions using VE as a teaching tool, what to look for in terms of human-centered systems and components, and current training uses in the Navy, Army, Air Force and Marines. Game-based and long distance training are explained, as are particular challenges such as the emergence of VE sickness. Chapters also highlight the combination of VE and cybernetics, robotics and artificial intelligence.

"The book provides researchers and developers of online collaborative systems with approaches for effective and efficient means that would assist e-learning students and teachers with precise and relevant information regarding the ongoing online learning activity at both individual and group levels"--Provided by publisher.