Score Reporting Research And Applications

Score reporting research is no longer limited to the psychometric properties of scores and subscores. Today, it encompasses design and evaluation for particular audiences, appropriate use of assessment outcomes, the utility and cognitive affordances of graphical representations, interactive report systems, and more. By studying how audiences understand the intended messages conveyed by score reports, researchers and industry professionals can develop more effective mechanisms for interpreting and using assessment data. Score Reporting Research and Applications brings together experts who design and evaluate score reports in both K-12 and higher education contexts and who conduct foundational research in related areas. The first section covers foundational validity issues in the use and interpretation of test scores; design principles drawn from related areas including cognitive science, human-computer interaction, and data visualization; and research on presenting specific types of assessment information to various audiences. The second section presents real-world applications of score report design and evaluation and of the presentation of assessment information. Across ten chapters, this volume offers a comprehensive overview of new techniques and possibilities in score reporting. The Open Access version of this book, available at http://www.taylorfrancis.com, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives 4.0 license.

This book constitutes the refereed proceedings of the First International Conference on Adaptive Instructional Systems, AIS 2019, held in July 2019 as part of HCI International 2019 in Orlando, FL, USA. HCII 2019 received a total of 5029 submissions, of which 1275 papers and 209 posters were accepted for publication after a careful reviewing process. The 50 papers presented in this volume are organized in topical sections named: Adaptive Instruction Design and Authoring, Interoperability and Standardization in Adaptive Instructional Systems, Instructional Theories in Adaptive Instruction, Learner Assessment and Modelling, AI in Adaptive Instructional Systems, Conversational Tutors.

Assessing Competencies for Social and Emotional Learning explores the conceptualization, development, and application of assessments of competencies and contextual factors related to social and emotional learning (SEL). As programs designed to teach students social and emotional competencies are being adopted at an ever‐increasing rate, new measurements are needed to understand their impact on student attitudes, behaviors, and academic performance. This book integrates standards of fairness, reliability, and validity, and lessons learned from personality and attitude assessment to facilitate the principled development and use of SEL assessments. Education professionals, assessment developers, and researchers will be better prepared to systematically develop and evaluate measures of social and emotional competencies.

This authoritative guide directs consumers and users of test scores on when and how to provide subscores and how to make informed decisions based on them. The book is designed to be accessible to practitioners and score users with varying levels of technical expertise, from executives of testing organizations and students who take tests to graduate students in educational measurement, psychometricians, and test developers. The theoretical background required to evaluate subscores and improve them are provided alongside examples of tests with subscores to illustrate their use and misuse. The first chapter covers the history of tests, subtests, scores, and subscores. Later chapters go into subscore reporting, evaluating and improving the quality of subscores, and alternatives to subscores when they are not appropriate. This thorough introduction to the existing research and best practices will be useful to graduate students, researchers, and practitioners.

This book on data visualization is the eighth in a planned series of books that examine key topics (e.g., learner modeling, instructional strategies, authoring, domain modeling, assessment, team tutoring, self-improving systems, data visualization, and competency based scenario design) in intelligent tutoring system (ITS) design. This book focuses on data visualization and how it is applied in ITSs. The chapters within this book specifically examine topics in relationship to the Generalized Intelligent Framework for Tutoring (GIFT) (Sottilare, Brawner, Goldberg & Holden, 2012; Sottilare, Brawner, Sinatra, & Johnston, 2017). GIFT is an open-source, domain-independent, modular, service-oriented architecture for ITSs. The design of GIFT allows for reusability, reduction in authoring time, and reducing the skill level needed to create an ITS. GIFT provides functionality to create ITSs, distribute ITSs to learners through the Cloud, conduct research to evaluate ITSs, and to examine instructional outcomes. Data visualization is an important topic for ITSs, as there are many different users of the systems (including learners, instructors, researchers, subject matter experts). The data that is collected by the ITS can be organized and displayed in a number of different ways. The current book includes a general discussion of how data visualizations can be applied in ITSs, as well as detailed specific examples of existing implementations, and technical details related to incorporating data visualization in ITSs. We believe this book can be used as a design tool for data visualization interfaces in ITSs.

This book introduces theories and practices for using assessment data to enhance learning and instruction. Topics include reshaping the homework review process, iterative learning engineering, learning progressions, learning maps, score report designing, the use of psychosocial data, and the combination of adaptive testing and adaptive learning. In addition, studies proposing new methods and strategies, technical details about the collection and maintenance of process data, and examples illustrating proposed methods and software are included. Chapters 1, 4, 6, 8, and 9 discuss how to make valid interpretations of results and achieve more efficient instructions from various sources of data. Chapters 3 and 7 propose and evaluate new methods to promote students’ learning by using evidence-based iterative learning engineering and supporting the teachers’ use of assessment data, respectively. Chapter 2 provides technical details on the collection, storage, and security protection of process data. Chapter 5 introduces software for automating some aspects of developmental education and the use of predictive modeling. Chapter 10 describes the barriers to using psychosocial data for formative assessment purposes. Chapter 11 describes a conceptual framework for adaptive learning and testing and gives an example of a functional learning and assessment system. In summary, the book includes comprehensive perspectives of the recent development and challenges of using test data for formative assessment purposes. The chapters provide innovative theoretical frameworks, new perspectives on the use of data with technology, and how to build new methods based on existing theories. This book is a useful resource to researchers who are interested in using data and technology to inform decision making, facilitate instructional utility, and achieve better learning outcomes.

The United States is formally represented around the world by approximately 14,000 Foreign Service officers and other personnel in the U.S. Department of State. Roughly one-third of them are required to be proficient in the local languages of the countries to which they are posted. To achieve this language proficiency for its staff, the State Department's Foreign Service Institute (FSI) provides intensive language instruction and assesses the proficiency of personnel before they are posted to a foreign country. The requirement for language proficiency is established in law and is incorporated in personnel decisions related to job placement, promotion, retention, and pay. A Principled Approach to Language Assessment: Considerations for the U.S. Foreign Service Institute evaluates the different approaches that exist to assess foreign language proficiency that FSI could potentially use. This report considers the key assessment approaches in the research literature that are appropriate for language testing, including, but not limited to, assessments that use task-based or performance-based approaches, adaptive online test administration, and portfolios.

Classroom Assessment and Educational Measurement explores the ways in which the theory and practice of both educational measurement and the assessment of student learning in classroom settings mutually inform one another. Chapters by assessment and measurement experts consider the nature of classroom assessment information, from student achievement to affective and socio-emotional attributes; how teachers interpret and work with assessment results; and emerging issues in assessment such as digital technologies and diversity/inclusion. This book uniquely considers the limitations of applying large-scale educational measurement theory to classroom assessment and the adaptations necessary to make this transfer useful. Researchers, graduate students, industry professionals, and policymakers will come away with an essential understanding of how the classroom assessment context is essential to broadening contemporary educational measurement perspectives. The Open Access version of this book, available at http://www.taylorfrancis.com, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives 4.0 license.