Embodied Conversational Agents

This book describes research in all aspects of the design, implementation, and evaluation of embodied conversational agents as well as details of specific working systems. Embodied conversational agents are computer-generated cartoonlike characters that demonstrate many of the same properties as humans in face-to-face conversation, including the ability to produce and respond to verbal and nonverbal communication. They constitute a type of (a) multimodal interface where the modalities are those natural to human conversation: speech, facial displays, hand gestures, and body stance; (b) software agent, insofar as they represent the computer in an interaction with a human or represent their human users in a computational environment (as avatars, for example); and (c) dialogue system where both verbal and nonverbal devices advance and regulate the dialogue between the user and the computer. With an embodied conversational agent, the visual dimension of interacting with an animated character on a screen plays an intrinsic role. Not just pretty pictures, the graphics display visual features of conversation in the same way that the face and hands do in face-to-face conversation among humans. This book describes research in all aspects of the design, implementation, and evaluation of embodied conversational agents as well as details of specific working systems. Many of the chapters are written by multidisciplinary teams of psychologists, linguists, computer scientists, artists, and researchers in interface design. The authors include Elisabeth Andre, Norm Badler, Gene Ball, Justine Cassell, Elizabeth Churchill, James Lester, Dominic Massaro, Cliff Nass, Sharon Oviatt, Isabella Poggi, Jeff Rickel, and Greg Sanders.

Embodied conversational agents (ECAs) are autonomous software entities with human-like appearance and communication skills. These agents can take on a number of different roles, for example, as an assistant, tutor, information provider, or customer service agent. They may also simply represent or entertain a user. The precise nature and benefits of different characteristics of ECAs requires careful investigation. Questions range from the function of an eyebrow raise to mechanisms for assessing and improving ECA trustworthiness. This book will help experts and designers in the specification and development of applications incorporating ECAs. Part 1 provides guidelines for evaluation methodologies and the identification of design and evaluation parameters. Part 2 demonstrates the importance of considering the user's perspective and interaction experience. Part 3 addresses issues in fine-tuning design parameters of ECAs and verifying the perceived effect. Finally, in Part 4 lessons learned from a number of application case studies are presented. The book is intended for both ECA researchers in academia and industry, and developers and designers interested in applying the technology.

This volume, containing the proceedings of IVA 2003, held at Kloster Irsee, in Germany, September 15–17, 2003, is testimony to the growing importance of IntelligentVirtualAgents(IVAs) asaresearch?eld.Wereceived67submissions, nearly twice as many as for IVA 2001, not only from European countries, but from China, Japan, and Korea, and both North and South America. As IVA research develops, a growing number of application areas and pl- forms are also being researched. Interface agents are used as part of larger - plications, often on the Web. Education applications draw on virtual actors and virtual drama, while the advent of 3D mobile computing and the convergence of telephones and PDAs produce geographically-aware guides and mobile - tertainment applications. A theme that will be apparent in a number of the papers in this volume is the impact of embodiment on IVA research – a char- teristic di?erentiating it to some extent from the larger ?eld of software agents.

The Handbook on Socially Interactive Agents provides a comprehensive overview of the research fields of Embodied Conversational Agents, Intelligent Virtual Agents, and Social Robotics. Socially Interactive Agents (SIAs), whether virtually or physically embodied, are autonomous agents that are able to perceive an environment including people or other agents, reason, decide how to interact, and express attitudes such as emotions, engagement, or empathy. They are capable of interacting with people and one another in a socially intelligent manner using multimodal communicative behaviors, with the goal to support humans in various domains. Written by international experts in their respective fields, the book summarizes research in the many important research communities pertinent for SIAs, while discussing current challenges and future directions. The handbook provides easy access to modeling and studying SIAs for researchers and students, and aims at further bridging the gap between the research communities involved. In two volumes, the book clearly structures the vast body of research. The first volume starts by introducing what is involved in SIAs research, in particular research methodologies and ethical implications of developing SIAs. It further examines research on appearance and behavior, focusing on multimodality. Finally, social cognition for SIAs is investigated using different theoretical models and phenomena such as theory of mind or pro-sociality. The second volume starts with perspectives on interaction, examined from different angles such as interaction in social space, group interaction, or long-term interaction. It also includes an extensive overview summarizing research and systems of human–agent platforms and of some of the major application areas of SIAs such as education, aging support, autism, and games.

Embodied Conversational Agents (ECAs) are a new type of computer interface which have human-like bodies and conversational skills. Users interacting with agents will be more engaged and participative if the agents exhibit behaviors which look coherent across different situations and emotional states. In the present work, we aim at increasing agents' believability by looking at two main aspects of the problem: - an agent must be able to show its emotional state and communicative intentions not only through specific facial expressions, gestures, etc., but also by varying the quality of its movements (e.g., their speed, amplitude, etc.) and the choice of the modalities that are used to communicate; - the agent must maintain a distinctive behavior, that is, its global behavior tendency has to remain apparent during any communication. We have developed a model to solve the two above issues. We have evaluated the realism and believability of the resulting agent's behaviors through perceptual tests and application scenario. The resulting system is highly extensible and configurable. It can also be used as a research tool to study human communication.

During a conversation, listeners do not assimilate passively all of the speaker's words; they actively participate in the interaction providing information about how they feel and what they think of the speaker's speech. The speaker relies on signals emitted by the listener to know if he is listening or not, understanding or not, agreeing or not, etc. This informs the speaker on the success or failure of the communication and helps him to decide how to carry on with the interaction. In this thesis, to refer to signals provided by listeners, we adopt the term backchannel proposed by Yngve. We define backchannels as acoustic and non-verbal signals provided during the speaker's turn to exchange information about the communicative functions: contact, perception, understanding, and attitude. Backchannels are emitted in a non-intrusive way: that is, without interrupting the speaker's speech. Two fundamental characteristics of backchannels are: (i) they can be emitted at different level of intentionality; (ii) they can be reactive (deriving from a first process of perception of the speaker's speech) or response (deriving from a more aware evaluation). A particular form of backchannel is the mimicry of the speaker's behavior. By mimicry, we mean the behavior displayed by an individual who does what another person does. This type of behavior has been proven to play quite an important role during conversations. Due to the importance of the listener's behavior, in this thesis we propose to implement a model that generates this type of behavior for an Embodied Conversational Agent while interacting with a user. We aim to improve the human-machine interaction.

"This book is a reference guide for researchers entering the promising field of conversational agents, providing an introduction to fundamental concepts in the field, collecting experiences of researchers working on conversational agents, and reviewing techniques for the design and application of conversational agents"--

The main topic of this volume is natural multimodal interaction. The book is unique in that it brings together a great many contributions regarding aspects of natural and multimodal interaction written by many of the important actors in the field. Topics addressed include talking heads, conversational agents, tutoring systems, multimodal communication, machine learning, architectures for multimodal dialogue systems, systems evaluation, and data annotation.