Toward Theoretically Meaningful Automated Essay Scoring

This volume provides a state-of-the-art overview of theory, methodology and practices in the assessment of writing. The focus throughout the book is on the construct of writing and its assessment: what constitutes writing ability and how can it be defined (in various contexts)? This question cannot be answered without looking into the methodological question of how to validate and measure the construct of writing ability. Throughout the book, therefore, discussions integrate theoretical and methodological issues. A number of chapters discusses whether varying definitions and varying operationalizations of writing ability are needed in various contexts, such as formative assessments versus summative assessments, large scale assessments versus individual assessments, different tasks, different genres, and different languages, but also different age groups. A range of rating methods is investigated and discussed in this book. The ongoing debate on holistic versus analytic ratings, and the different underlying conceptions of writing proficiency, is a pertinent matter, on which a number of chapters in this volume shed new light. The matter is discussed and analyzed from various angles, such as generalizability of judgements and usability in formative contexts. Another fundamental debate concerns computer scoring of written products. A nuanced discussion of its validity is presented in this volume.

In an era of curricular changes and experiments and high-stakes testing, educational measurement and evaluation is more important than ever. In addition to expected entries covering the basics of traditional theories and methods, other entries discuss important sociopolitical issues and trends influencing the future of that research and practice. Textbooks, handbooks, monographs and other publications focus on various aspects of educational research, measurement and evaluation, but to date, there exists no major reference guide for students new to the field. This comprehensive work fills that gap, covering traditional areas while pointing the way to future developments. Features: Nearly 700 signed entries are contained in an authoritative work spanning four volumes and available in choice of electronic and/or print formats. Although organized A-to-Z, front matter includes a Reader’s Guide grouping entries thematically to help students interested in a specific aspect of education research, measurement, and evaluation to more easily locate directly related entries. (For instance, sample themes include Data, Evaluation, Measurement Concepts & Issues, Research, Sociopolitical Issues, Standards.) Back matter includes a Chronology of the development of the field; a Resource Guide to classic books, journals, and associations; and a detailed Index. Entries conclude with References/Further Readings and Cross References to related entries. The Index, Reader’s Guide themes, and Cross References will combine to provide robust search-and-browse in the e-version.

Presents a collection of essays discussing the theories and models of writing research.

In the edited volume Writing for Professional Development, Giulia Ortoleva, Mireille Bétrancourt and Stephen Billett explore the relation between writing and professional development. Two main perspectives are considered: learning to write professionally and writing to learn the profession.

This new volume is the first to focus entirely on automated essay scoring and evaluation. It is intended to provide a comprehensive overview of the evolution and state-of-the-art of automated essay scoring and evaluation technology across several disciplines, including education, testing and measurement, cognitive science, computer science, and computational linguistics. The development of this technology has led to many questions and concerns. Automated Essay Scoring attempts to address some of these questions including: *How can automated scoring and evaluation supplement classroom instruction? *How does the technology actually work? *Can it improve students' writing? *How reliable is the technology? *How can these computing methods be used to develop evaluation tools? *What are the state-of the-art essay evaluation technologies and automated scoring systems? Divided into four parts, the first part reviews the teaching of writing and how computers can contribute to it. Part II analyzes actual automated essay scorers including e-raterTM, Intellimetric, and the Intelligent Essay Assessor. The third part analyzes related psychometric issues, and the final part reviews innovations in the field. This book is ideal for researchers and advanced students interested in automated essay scoring from the fields of testing and measurement, education, cognitive science, language, and computational linguistics.

This book discusses the state of the art of automated essay scoring, its challenges and its potential. One of the earliest applications of artificial intelligence to language data (along with machine translation and speech recognition), automated essay scoring has evolved to become both a revenue-generating industry and a vast field of research, with many subfields and connections to other NLP tasks. In this book, we review the developments in this field against the backdrop of Elias Page's seminal 1966 paper titled "The Imminence of Grading Essays by Computer." Part 1 establishes what automated essay scoring is about, why it exists, where the technology stands, and what are some of the main issues. In Part 2, the book presents guided exercises to illustrate how one would go about building and evaluating a simple automated scoring system, while Part 3 offers readers a survey of the literature on different types of scoring models, the aspects of essay quality studied in prior research, and the implementation and evaluation of a scoring engine. Part 4 offers a broader view of the field inclusive of some neighboring areas, and Part \ref{part5} closes with summary and discussion. This book grew out of a week-long course on automated evaluation of language production at the North American Summer School for Logic, Language, and Information (NASSLLI), attended by advanced undergraduates and early-stage graduate students from a variety of disciplines. Teachers of natural language processing, in particular, will find that the book offers a useful foundation for a supplemental module on automated scoring. Professionals and students in linguistics, applied linguistics, educational technology, and other related disciplines will also find the material here useful.