The Evaluation Game

This text examines the origin and effects of publication metrics that are used to measure academic productivity around the world.

Scientific research is communicated, organized, financed, governed, and evaluated through the process of publication. The result of this process is a highly competitive academic environment that rewards researchers for high volume publication, preferably in high-impact journals, leading to the popularised expression 'publish or perish'. Universities and other scientific institutions are under similar pressure, with their aggregated research output being under constant scrutiny. This innovative text provides a detailed introduction to the origin and development of the scholarly metrics used to measure academic productivity, and the effect they have upon the quality and diversity of scientific research. With its careful attention to both the positive and negative outcomes of research evaluation and their distinct expressions around the globe, The Evaluation Game guides the way to a more grounded understanding of metrics, and the diverse academic cultures they give rise to.

It was a pleasure to provide an introduction to a new volume on user experience evaluation in games. The scope, depth, and diversity of the work here is amazing. It attests to the growing popularity of games and the increasing importance developing a range of theories, methods, and scales to evaluate them. This evolution is driven by the cost and complexity of games being developed today. It is also driven by the need to broaden the appeal of games. Many of the approaches described here are enabled by new tools and techniques. This book (along with a few others) represents a watershed in game evaluation and understanding. The eld of game evaluation has truly “come of age”. The broader eld of HCI can begin to look toward game evaluation for fresh, critical, and sophisticated thi- ing about design evaluation and product development. They can also look to games for groundbreaking case studies of evaluation of products. I’ll brie y summarize each chapter below and provide some commentary. In conclusion, I will mention a few common themes and offer some challenges. Discussion In Chapter 1, User Experience Evaluation in Entertainment, Bernhaupt gives an overview and presents a general framework on methods currently used for user experience evaluation. The methods presented in the following chapters are s- marized and thus allow the reader to quickly assess the right set of methods that will help to evaluate the game under development.

Evaluating interactive systems for their user experience (UX) is a standard approach in industry and research today. This book explores the areas of game design and development and Human Computer Interaction (HCI) as ways to understand the various contributing aspects of the overall gaming experience. Fully updated, extended and revised this book is based upon the original publication Evaluating User Experience in Games, and provides updated methods and approaches ranging from user- orientated methods to game specific approaches. New and emerging methods and areas explored include physiologically- orientated UX evaluation, user behaviour, telemetry based methods and social play as effective evaluation techniques for gaming design and evolving user-experience. Game User Experience Evaluation allows researchers, PhD students as well as game designers and developers to get an overview on available methods for all stages of the development life cycle.

It was a pleasure to provide an introduction to a new volume on user experience evaluation in games. The scope, depth, and diversity of the work here is amazing. It attests to the growing popularity of games and the increasing importance developing a range of theories, methods, and scales to evaluate them. This evolution is driven by the cost and complexity of games being developed today. It is also driven by the need to broaden the appeal of games. Many of the approaches described here are enabled by new tools and techniques. This book (along with a few others) represents a watershed in game evaluation and understanding. The eld of game evaluation has truly “come of age”. The broader eld of HCI can begin to look toward game evaluation for fresh, critical, and sophisticated thi- ing about design evaluation and product development. They can also look to games for groundbreaking case studies of evaluation of products. I’ll brie y summarize each chapter below and provide some commentary. In conclusion, I will mention a few common themes and offer some challenges. Discussion In Chapter 1, User Experience Evaluation in Entertainment, Bernhaupt gives an overview and presents a general framework on methods currently used for user experience evaluation. The methods presented in the following chapters are s- marized and thus allow the reader to quickly assess the right set of methods that will help to evaluate the game under development.

In an increasingly scientific and technological world the need for a knowledgeable citizenry, individuals who understand the fundamentals of technological ideas and think critically about these issues, has never been greater. There is growing appreciation across the broader education community that educational three dimensional virtual learning environments are part of the daily lives of citizens, not only regularly occurring in schools and in after-school programs, but also in informal settings like museums, science centers, zoos and aquariums, at home with family, in the workplace, during leisure time when children and adults participate in community-based activities. This blurring of the boundaries of where, when, why, how and with whom people learn, along with better understandings of learning as a personally constructed, life-long process of making meaning and shaping identity, has initiated a growing awareness in the field that the questions and frameworks guiding assessing these environments should be reconsidered in light of these new realities. The audience for this book will be researchers working in the Serious Games arena along with distance education instructors and administrators and students on the cutting edge of assessment in computer generated environments.

"Two exercises using a Game-Based Simulation (GBS) were conducted by the U.S. Army Research Development and Engineering Command, Simulation and Training Technology Center (RDECOM-STTC) and the United Kingdom Land Warfare Development Group. Soldiers from the U.S. Army and the U.K. military conducted coalition mission rehearsals during each exercise. Data were collected on the system user interface, on the effectiveness of unit and joint exercise sessions, and on After Action Review (AAR) functionality and applications. Several issues in technological capabilities limited and constrained the military tasks that could be performed during the exercises, and limited the AARs. Nevertheless, questionnaire data collected during each exercise indicated several positive aspects of using game-based simulations. The GBS system was considered capable of providing considerable scope for general dismounted Soldier rehearsal and training. The graphics and user interface were judged adequate for use in training rehearsals and AARs, especially in preparation for home station field training exercises. The largest negative issue was the limited number of weapon types and equipment. The second largest issue was the limited equipment functionality that the system supported. A third issue was the lack of sufficient numbers of civilians and opposing forces for different interactions in the non-kinetic exercises."--DTIC.

"In this MacArthur Report, Suhr explores digital music communities' use of evaluation games and digital badges for both casual music evaluators and musicians. The first case study focuses on audience evaluation via playing Spotify's Hit or Not game, where game players evaluate a song's hit potential. Suhr explores the gamification of learning and evaluation; when the act of music evaluation turns into a game, what do the players learn, and what may be the implications of this type of evaluation? The second, more in-depth case study on Indaba Music examines the process of gaining badges through involvement in contests. Suhr questions whether or not gaining badges holds significance for musicians. Taken as a whole, this report analyzes how digital badges are perceived by both music listeners and musicians. To what extent can digital badges be an effective way to represent and credit musicians' accomplishments and merits? What are the emerging challenges, benefits, and shortcomings in the use of digital badges as an alternative evaluation mechanism? How do the uses of digital badges in the context of assessing creativity intersect or diverge with the practices related to education and other non-creative fields?"--Provided by publisher.