User Centered Evaluation Of Visual Analytics

Visual analytics has come a long way since its inception in 2005. The amount of data in the world today has increased significantly and experts in many domains are struggling to make sense of their data. Visual analytics is helping them conduct their analyses. While software developers have worked for many years to develop software that helps users do their tasks, this task is becoming more and more onerous, as understanding the needs and data used by expert users requires more than some simple usability testing during the development process. The need for a user centered evaluation process was envisioned in Illuminating the Path, the seminal work on visual analytics by James Thomas and Kristin Cook in 2005. We have learned over the intervening years that not only will user-centered evaluation help software developers to turn out products that have more utility, the evaluation efforts can also help point out the direction for future research efforts. This book describes the efforts that go into analysis, including critical thinking, sensemaking, and various analytics techniques learned from the intelligence community. Support for these components is needed in order to provide the most utility for the expert users. There are a good number of techniques for evaluating software that has been developed within the human-computer interaction (HCI) community. While some of these techniques can be used as is, others require modifications. These too are described in the book. An essential point to stress is that the users of the domains for which visual analytics tools are being designed need to be involved in the process. The work they do and the obstacles in their current processes need to be understood in order to determine both the types of evaluations needed and the metrics to use in these evaluations. At this point in time, very few published efforts describe more than informal evaluations. The purpose of this book is to help readers understand the need for more user-centered evaluations to drive both better-designed products and to define areas for future research. Hopefully readers will view this work as an exciting and creative effort and will join the community involved in these efforts.

Visual analytics has come a long way since its inception in 2005. The amount of data in the world today has increased significantly and experts in many domains are struggling to make sense of their data. Visual analytics is helping them conduct their analyses. While software developers have worked for many years to develop software that helps users do their tasks, this task is becoming more and more onerous, as understanding the needs and data used by expert users requires more than some simple usability testing during the development process. The need for a user-centered evaluation process was envisioned in Illuminating the Path, the seminal work on visual analytics by James Thomas and Kristin Cook in 2005. We have learned over the intervening years that not only will user-centered evaluation help software developers to turn out products that have more utility, the evaluation efforts can also help point out the direction for future research efforts. This book describes the efforts that go into analysis, including critical thinking, sensemaking, and various analytics techniques learned from the intelligence community. Support for these components is needed in order to provide the most utility for the expert users. There are a good number of techniques for evaluating software that hasbeen developed within the human-computer interaction (HCI) community. While some of these techniques can be used as is, others require modifications. These too are described in the book. An essential point to stress is that the users of the domains for which visual analytics tools are being designed need to be involved in the process. The work they do and the obstacles in their current processes need to be understood in order to determine both the types of evaluations needed and the metrics to use in these evaluations. At this point in time, very few published efforts describe more than informal evaluations. The purpose of this book is to help readers understand the need for more user-centered evaluations to drive both better-designed products and to define areas for future research. Hopefully readers will view this work as an exciting and creative effort and will join the community involved in these efforts.

The primary goal for collecting and analyzing temporal data differs between individuals and their domain of expertise e.g., forecasting might be the goal in meteorology, anomaly detection might be the goal in finance. While the goal differs, one common denominator is the need for exploratory analysis of the temporal data, as this can aid the search for useful information. However, as temporal data can be challenging to understand and visualize, selecting appropriate visual representations for the domain and data at hand becomes a challenge. Moreover, many visual representations can show a single variable that changes over time, displaying multiple variables in a clear and easily accessible way is much harder, and inference-making and pattern recognition often require visualization of multiple variables. Additionally, as visualization aims to gain insight, it becomes crucial to investigate whether the representations used help users gain this insight. Furthermore, to create effective and efficient visual analysis tools, it is vital to understand the structure of the data, how this data can be represented, and have a clear understanding of the user needs. Developing useful visual representations can be challenging, but through close collaboration and involvement of end-users in the entire process, useful results can be accomplished. This thesis aims to investigate the usability of different visual representations for different types of multivariate temporal data, users, and tasks. Five user studies have been conducted to investigate different representation spaces, layouts, and interaction methods for investigating representations’ ability to facilitate users when analyzing and exploring such temporal datasets. The first study investigated and evaluated the experience of different radial design ideas for finding and comparison tasks when presenting hourly data based on an analog clock metaphor. The second study investigated 2D and 3D parallel coordinates for pattern finding. In the third study, the usability of three linear visual representations for presenting indoor climate data was investigated with domain experts. The fourth study continued on the third study and developed and evaluated a visual analytics tool with different visual representations and interaction techniques with domain experts. Finally, in the fifth study, another visual analytics tool presenting visual representations of temporal data was developed and evaluated with domain experts working and conducting experiments in Antarctica. The research conducted within the scope of this thesis concludes that it is vital to understand the characteristics of the temporal data and user needs for selecting the optimal representations. Without this knowledge, it becomes much harder to choose visual representations to help users gain insight from the data. It is also crucial to evaluate the perception and usability of the chosen visual representations.

The field of computer graphics combines display hardware, software, and interactive techniques in order to display and interact with data generated by applications. Visualization is concerned with exploring data and information graphically in such a way as to gain information from the data and determine significance. Visual analytics is the science of analytical reasoning facilitated by interactive visual interfaces. Expanding the Frontiers of Visual Analytics and Visualization provides a review of the state of the art in computer graphics, visualization, and visual analytics by researchers and developers who are closely involved in pioneering the latest advances in the field. It is a unique presentation of multi-disciplinary aspects in visualization and visual analytics, architecture and displays, augmented reality, the use of color, user interfaces and cognitive aspects, and technology transfer. It provides readers with insights into the latest developments in areas such as new displays and new display processors, new collaboration technologies, the role of visual, multimedia, and multimodal user interfaces, visual analysis at extreme scale, and adaptive visualization.

The emergence of multilayer networks as a concept from the field of complex systems provides many new opportunities for the visualization of network complexity, and has also raised many new exciting challenges. The multilayer network model recognizes that the complexity of relationships between entities in real-world systems is better embraced as several interdependent subsystems (or layers) rather than a simple graph approach. Despite only recently being formalized and defined, this model can be applied to problems in the domains of life sciences, sociology, digital humanities, and more. Within the domain of network visualization there already are many existing systems, which visualize data sets having many characteristics of multilayer networks, and many techniques, which are applicable to their visualization. In this Synthesis Lecture, we provide an overview and structured analysis of contemporary multilayer network visualization. This is not only for researchers in visualization, but also for those who aim to visualize multilayer networks in the domain of complex systems, as well as those solving problems within application domains. We have explored the visualization literature to survey visualization techniques suitable for multilayer network visualization, as well as tools, tasks, and analytic techniques from within application domains. We also identify the research opportunities and examine outstanding challenges for multilayer network visualization along with potential solutions and future research directions for addressing them.

Data science addresses the need to extract knowledge and information from data volumes, often from real-time sources in a wide variety of disciplines such as astronomy, bioinformatics, engineering, science, medicine, social science, business, and the humanities. The range and volume of data sources has increased enormously over time, particularly those generating real-time data. This has posed additional challenges for data management and data analysis of the data and effective representation and display. A wide range of application areas are able to benefit from the latest visual tools and facilities. Rapid analysis is needed in areas where immediate decisions need to be made. Such areas include weather forecasting, the stock exchange, and security threats. In areas where the volume of data being produced far exceeds the current capacity to analyze all of it, attention is being focussed how best to address these challenges. Optimum ways of addressing large data sets across a variety of disciplines have led to the formation of national and institutional Data Science Institutes and Centers. Being driven by national priority, they are able to attract support for research and development within their organizations and institutions to bring together interdisciplinary expertise to address a wide variety of problems. Visual computing is a set of tools and methodologies that utilize 2D and 3D images to extract information from data. Such methods include data analysis, simulation, and interactive exploration. These are analyzed and discussed.

At the 2016 IEEE VIS Conference in Baltimore, Maryland, a panel of experts from the Scientific Visualization (SciVis) community gathered to discuss why the SciVis component of the conference had been shrinking significantly for over a decade. As the panelists concluded and opened the session to questions from the audience, Annie Preston, a Ph.D. student at the University of California, Davis, asked whether the panelists thought diversity or, more specifically, the lack of diversity was a factor. This comment ignited a lively discussion of diversity: not only its impact on Scientific Visualization, but also its role in the visualization community at large. The goal of this book is to expand and organize the conversation. In particular, this book seeks to frame the diversity and inclusion topic within the Visualization community, illuminate the issues, and serve as a starting point to address how to make this community more diverse and inclusive. This book acknowledges that diversity is a broad topic with many possible meanings. Expanded definitions of diversity that are relevant to the Visualization community and to computing at large are considered. The broader conversation of inclusion and diversity is framed within the broader sociological context in which it must be considered. Solutions to recruit and retain a diverse research community and strategies for supporting inclusion efforts are presented. Additionally, community members present short stories detailing their ""non-inclusive"" experiences in an effort to facilitate a community-wide conversation surrounding very difficult situations. It is important to note that this is by no means intended to be a comprehensive, authoritative statement on the topic. Rather, this book is intended to open the conversation and begin to build a framework for diversity and inclusion in this specific research community. While intended for the Visualization community, ideally, this book will provide guidance for any computing community struggling with similar issues and looking for solutions.

There is ample evidence in the visualization community that individual differences matter. These prior works highlight various personality traits and cognitive abilities that can modulate the use of the visualization systems and demonstrate a measurable influence on speed, accuracy, process, and attention. Perhaps the most important implication of this body of work is that we can use individual differences as a mechanism for estimating when a design is effective or to identify when people may struggle with visualization designs. These effects can have a critical impact on consequential decision-making processes. One study that appears in this book investigated the impact of visualization on medical decision-making showed that visual aides tended to be most beneficial for people with high spatial ability, a metric that measures a person’s ability to represent and manipulate two- or three-dimensional representations of objects mentally. The results showed that participants with low spatial ability had difficulty interpreting and analyzing the underlying medical data when they use visual aids. Overall, approximately 50% of the studied population were unsupported by the visualization tools when making a potentially life-critical decision. As data fluency continues to become an essential skill for our everyday lives, we must embrace the growing need to understand the factors that may render our tools ineffective and identify concrete steps for improvement. This book presents my current understanding of how individual differences in personality interact with visualization use and draws from recent research in the Visualization, Human-Computer Interaction, and Psychology communities. We focus on the specific designs and tasks for which there is concrete evidence of performance divergence due to personality. Additionally, we highlight an exciting research agenda that is centered around creating tailored visualization systems that are aligned with people’s abilities. The purpose of this book is to underscore the need to consider individual differences when designing and evaluating visualization systems and to call attention to this critical research direction.