Computerized Adaptive And Multistage Testing With R

The goal of this guide and manual is to provide a practical and brief overview of the theory on computerized adaptive testing (CAT) and multistage testing (MST) and to illustrate the methodologies and applications using R open source language and several data examples. Implementation relies on the R packages catR and mstR that have been already or are being developed by the first author (with the team) and that include some of the newest research algorithms on the topic. The book covers many topics along with the R-code: the basics of R, theoretical overview of CAT and MST, CAT designs, CAT assembly methodologies, CAT simulations, catR package, CAT applications, MST designs, IRT-based MST methodologies, tree-based MST methodologies, mstR package, and MST applications. CAT has been used in many large-scale assessments over recent decades, and MST has become very popular in recent years. R open source language also has become one of the most useful tools for applications in almost all fields, including business and education. Though very useful and popular, R is a difficult language to learn, with a steep learning curve. Given the obvious need for but with the complex implementation of CAT and MST, it is very difficult for users to simulate or implement CAT and MST. Until this manual, there has been no book for users to design and use CAT and MST easily and without expense; i.e., by using the free R software. All examples and illustrations are generated using predefined scripts in R language, available for free download from the book's website.

Unlike other forms of adaptive testing, multistage testing (MST) is highly suitable for testing educational achievement because it can be adapted to educational surveys and student testing. This volume provides the first unified source of information on the design, psychometrics, implementation, and operational use of MST. It shows how to apply theoretical statistical tools to testing in novel and useful ways. It also explains how to explicitly tie the assumptions made by each model to observable (or at least inferable) data conditions.

This book offers a comprehensive introduction to the latest developments in the theory and practice of CAT. It can be used both as a basic reference and a valuable resource on test theory. It covers such topics as item selection and ability estimation, item pool development and maintenance, item calibration and model fit, and testlet-based adaptive testing, as well as the operational aspects of existing large-scale CAT programs.

The general theme of this book is to present the applications of artificial intelligence (AI) in test development. In particular, this book includes research and successful examples of using AI technology in automated item generation, automated test assembly, automated scoring, and computerized adaptive testing. By utilizing artificial intelligence, the efficiency of item development, test form construction, test delivery, and scoring could be dramatically increased. Chapters on automated item generation offer different perspectives related to generating a large number of items with controlled psychometric properties including the latest development of using machine learning methods. Automated scoring is illustrated for different types of assessments such as speaking and writing from both methodological aspects and practical considerations. Further, automated test assembly is elaborated for the conventional linear tests from both classical test theory and item response theory perspectives. Item pool design and assembly for the linear-on-the-fly tests elaborates more complications in practice when test security is a big concern. Finally, several chapters focus on computerized adaptive testing (CAT) at either item or module levels. CAT is further illustrated as an effective approach to increasing test-takers’ engagement in testing. In summary, the book includes both theoretical, methodological, and applied research and practices that serve as the foundation for future development. These chapters provide illustrations of efforts to automate the process of test development. While some of these automation processes have become common practices such as automated test assembly, automated scoring, and computerized adaptive testing, some others such as automated item generation calls for more research and exploration. When new AI methods are emerging and evolving, it is expected that researchers can expand and improve the methods for automating different steps in test development to enhance the automation features and practitioners can adopt quality automation procedures to improve assessment practices.

The arrival of the computer in educational and psychological testing has led to the current popularity of adaptive testing---a testing format in which the computer uses statistical information about the test items to automatically adapt their selection to a real-time update of the test taker’s ability estimate. This book covers such key features of adaptive testing as item selection and ability estimation, adaptive testing with multidimensional abilities, sequencing adaptive test batteries, multistage adaptive testing, item-pool design and maintenance, estimation of item and item-family parameters, item and person fit, as well as adaptive mastery and classification testing. It also shows how these features are used in the daily operations of several large-scale adaptive testing programs.

Used worldwide in assessment and professional certification contexts, computerized adaptive testing (CAT) offers a powerful means to measure individual differences or make classifications. This authoritative work from CAT pioneer David J. Weiss and Alper Şahin provides a complete how-to guide for planning and implementing an effective CAT to create a test unique to each person in real time. The book reviews the history of CAT and the basics of item response theory used in CAT. It walks the reader through developing an item bank, pretesting and linking items, selecting required CAT options, and using simulations to design a CAT. Available software for CAT delivery is described, including links to free and commercial options. Engaging multidisciplinary examples illustrate applications of CAT for measuring ability, achievement, proficiency, personality, attitudes, perceptions, patients’ reports of their symptoms, and academic or clinical progress.

The eight-volume set LNCS 13375 – 13382 constitutes the proceedings of the 22nd International Conference on Computational Science and Its Applications, ICCSA 2022, which was held in Malaga, Spain during July 4 – 7, 2022. The first two volumes contain the proceedings from ICCSA 2022, which are the 57 full and 24 short papers presented in these books were carefully reviewed and selected from 279 submissions. The other six volumes present the workshop proceedings, containing 285 papers out of 815 submissions. These six volumes includes the proceedings of the following workshops: ​ Advances in Artificial Intelligence Learning Technologies: Blended Learning, STEM, Computational Thinking and Coding (AAILT 2022); Workshop on Advancements in Applied Machine-learning and Data Analytics (AAMDA 2022); Advances in information Systems and Technologies for Emergency management, risk assessment and mitigation based on the Resilience (ASTER 2022); Advances in Web Based Learning (AWBL 2022); Blockchain and Distributed Ledgers: Technologies and Applications (BDLTA 2022); Bio and Neuro inspired Computing and Applications (BIONCA 2022); Configurational Analysis For Cities (CA Cities 2022); Computational and Applied Mathematics (CAM 2022), Computational and Applied Statistics (CAS 2022); Computational Mathematics, Statistics and Information Management (CMSIM); Computational Optimization and Applications (COA 2022); Computational Astrochemistry (CompAstro 2022); Computational methods for porous geomaterials (CompPor 2022); Computational Approaches for Smart, Conscious Cities (CASCC 2022); Cities, Technologies and Planning (CTP 2022); Digital Sustainability and Circular Economy (DiSCE 2022); Econometrics and Multidimensional Evaluation in Urban Environment (EMEUE 2022); Ethical AI applications for a human-centered cyber society (EthicAI 2022); Future Computing System Technologies and Applications (FiSTA 2022); Geographical Computing and Remote Sensing for Archaeology (GCRSArcheo 2022); Geodesign in Decision Making: meta planning and collaborative design for sustainable and inclusive development (GDM 2022); Geomatics in Agriculture and Forestry: new advances and perspectives (GeoForAgr 2022); Geographical Analysis, Urban Modeling, Spatial Statistics (Geog-An-Mod 2022); Geomatics for Resource Monitoring and Management (GRMM 2022); International Workshop on Information and Knowledge in the Internet of Things (IKIT 2022); 13th International Symposium on Software Quality (ISSQ 2022); Land Use monitoring for Sustanability (LUMS 2022); Machine Learning for Space and Earth Observation Data (MALSEOD 2022); Building multi-dimensional models for assessing complex environmental systems (MES 2022); MOdels and indicators for assessing and measuring the urban settlement deVElopment in the view of ZERO net land take by 2050 (MOVEto0 2022); Modelling Post-Covid cities (MPCC 2022); Ecosystem Services: nature’s contribution to people in practice. Assessment frameworks, models, mapping, and implications (NC2P 2022); New Mobility Choices For Sustainable and Alternative Scenarios (NEMOB 2022); 2nd Workshop on Privacy in the Cloud/Edge/IoT World (PCEIoT 2022); Psycho-Social Analysis of Sustainable Mobility in The Pre- and Post-Pandemic Phase (PSYCHE 2022); Processes, methods and tools towards RESilient cities and cultural heritage prone to SOD and ROD disasters (RES 2022); Scientific Computing Infrastructure (SCI 2022); Socio-Economic and Environmental Models for Land Use Management (SEMLUM 2022); 14th International Symposium on Software Engineering Processes and Applications (SEPA 2022); Ports of the future - smartness and sustainability (SmartPorts 2022); Smart Tourism (SmartTourism 2022); Sustainability Performance Assessment: models, approaches and applications toward interdisciplinary and integrated solutions (SPA 2022); Specifics of smart cities development in Europe (SPEED 2022); Smart and Sustainable Island Communities (SSIC 2022); Theoretical and Computational Chemistryand its Applications (TCCMA 2022); Transport Infrastructures for Smart Cities (TISC 2022); 14th International Workshop on Tools and Techniques in Software Development Process (TTSDP 2022); International Workshop on Urban Form Studies (UForm 2022); Urban Regeneration: Innovative Tools and Evaluation Model (URITEM 2022); International Workshop on Urban Space and Mobilities (USAM 2022); Virtual and Augmented Reality and Applications (VRA 2022); Advanced and Computational Methods for Earth Science Applications (WACM4ES 2022); Advanced Mathematics and Computing Methods in Complex Computational Systems (WAMCM 2022).

This book introduces theories and practices for using assessment data to enhance learning and instruction. Topics include reshaping the homework review process, iterative learning engineering, learning progressions, learning maps, score report designing, the use of psychosocial data, and the combination of adaptive testing and adaptive learning. In addition, studies proposing new methods and strategies, technical details about the collection and maintenance of process data, and examples illustrating proposed methods and software are included. Chapters 1, 4, 6, 8, and 9 discuss how to make valid interpretations of results and achieve more efficient instructions from various sources of data. Chapters 3 and 7 propose and evaluate new methods to promote students’ learning by using evidence-based iterative learning engineering and supporting the teachers’ use of assessment data, respectively. Chapter 2 provides technical details on the collection, storage, and security protection of process data. Chapter 5 introduces software for automating some aspects of developmental education and the use of predictive modeling. Chapter 10 describes the barriers to using psychosocial data for formative assessment purposes. Chapter 11 describes a conceptual framework for adaptive learning and testing and gives an example of a functional learning and assessment system. In summary, the book includes comprehensive perspectives of the recent development and challenges of using test data for formative assessment purposes. The chapters provide innovative theoretical frameworks, new perspectives on the use of data with technology, and how to build new methods based on existing theories. This book is a useful resource to researchers who are interested in using data and technology to inform decision making, facilitate instructional utility, and achieve better learning outcomes.