Vision Based Interaction

In its early years, the field of computer vision was largely motivated by researchers seeking computational models of biological vision and solutions to practical problems in manufacturing, defense, and medicine. For the past two decades or so, there has been an increasing interest in computer vision as an input modality in the context of human-computer interaction. Such vision-based interaction can endow interactive systems with visual capabilities similar to those important to human-human interaction, in order to perceive non-verbal cues and incorporate this information in applications such as interactive gaming, visualization, art installations, intelligent agent interaction, and various kinds of command and control tasks. Enabling this kind of rich, visual and multimodal interaction requires interactive-time solutions to problems such as detecting and recognizing faces and facial expressions, determining a person's direction of gaze and focus of attention, tracking movement of the body, and recognizing various kinds of gestures. In building technologies for vision-based interaction, there are choices to be made as to the range of possible sensors employed (e.g., single camera, stereo rig, depth camera), the precision and granularity of the desired outputs, the mobility of the solution, usability issues, etc. Practical considerations dictate that there is not a one-size-fits-all solution to the variety of interaction scenarios; however, there are principles and methodological approaches common to a wide range of problems in the domain. While new sensors such as the Microsoft Kinect are having a major influence on the research and practice of vision-based interaction in various settings, they are just a starting point for continued progress in the area. In this book, we discuss the landscape of history, opportunities, and challenges in this area of vision-based interaction; we review the state-of-the-art and seminal works in detecting and recognizing the human body and its components; we explore both static and dynamic approaches to "looking at people" vision problems; and we place the computer vision work in the context of other modalities and multimodal applications. Readers should gain a thorough understanding of current and future possibilities of computer vision technologies in the context of human-computer interaction.

This book constitutes the thoroughly refereed post-proceedings of the 7th International Workshop on Gesture-Based Human-Computer Interaction and Simulation, GW 2007, held in Lisbon, Portugal, in May 2007. The 31 revised papers presented were carefully selected from 53 submissions. The papers are organized in topical sections on analysis and synthesis of gesture; theoretical aspects of gestural communication and interaction; vision-based gesture recognition; sign language processing; gesturing with tangible interfaces and in virtual and augmented reality; gesture for music and performing arts; gesture for therapy and rehabilitation; and gesture in Mobile computing and usability studies.

200Ts Vision of Vision One of my formative childhood experiences was in 1968 stepping into the Uptown Theater on Connecticut Avenue in Washington, DC, one of the few movie theaters nationwide that projected in large-screen cinerama. I was there at the urging of a friend, who said I simply must see the remarkable film whose run had started the previous week. "You won't understand it," he said, "but that doesn't matter. " All I knew was that the film was about science fiction and had great special eflPects. So I sat in the front row of the balcony, munched my popcorn, sat back, and experienced what was widely touted as "the ultimate trip:" 2001: A Space Odyssey. My friend was right: I didn't understand it. . . but in some senses that didn't matter. (Even today, after seeing the film 40 times, I continue to discover its many subtle secrets. ) I just had the sense that I had experienced a creation of the highest aesthetic order: unique, fresh, awe inspiring. Here was a film so distinctive that the first half hour had no words whatsoever; the last half hour had no words either; and nearly all the words in between were banal and irrelevant to the plot - quips about security through Voiceprint identification, how to make a phonecall from a space station, government pension plans, and so on.

Leading scientists describe how advances in computer vision can change how we interact with computers.

Vision-based human-computer interaction means to use computer-vision technology for interaction of a user with a computer-based application. This idea has recently found particular interest of research. Among the many possibilities of implementing interaction, we focus on hand-based interaction, expressed by single hand postures, sequences of hand postures, and pointing. Two system architectures are presented which address different scenarios of interaction, and which establish the frame for several problems for which solutions are worked out. The system ZYKLOP treats hand gestures performed in a local environment, for example, on a limited area of the table-top. The goal with respect to this classical scenario is a more reliable system behaviour. Contributions concern color-based segmentation, forearm-hand separation as a precondition to more shape-based hand gesture classification, and classification of static and dynamic gestures. The ARGUS concept makes a first step towards the systematic analysis of hand gesture based interaction combined with pointing in a spatial environment with sensitive regions.Special topics addressed within the architectural framework of ARGUS include the recognition of details from the distance, compensation of varying illumination, changing orientation of the hand with respect to the cameras, estimation of pointing directions, and object recognition.

This book constitutes the thoroughly refereed post-proceedings of the International Gesture Workshop, GW'99, held in Gif-sur-Yvette, France, in March 1999. The 16 revised long papers and seven revised short papers were carefully reviewed for inclusion in the book. Also included are four invited papers and the transcription of a round table discussion. The papers are organized in sections on human perception and production of gesture, localization and segmentation, recognition, sign language, gesture synthesis and animation, and multimodality.

Research on the multifaceted aspects of modeling, analysis, and synthesis of - man gesture is receiving growing interest from both the academic and industrial communities. On one hand, recent scienti?c developments on cognition, on - fect/emotion, on multimodal interfaces, and on multimedia have opened new perspectives on the integration of more sophisticated models of gesture in c- putersystems.Ontheotherhand,theconsolidationofnewtechnologiesenabling “disappearing” computers and (multimodal) interfaces to be integrated into the natural environments of users are making it realistic to consider tackling the complex meaning and subtleties of human gesture in multimedia systems, - abling a deeper, user-centered, enhanced physical participation and experience in the human-machine interaction process. The research programs supported by the European Commission and s- eral national institutions and governments individuated in recent years strategic ?elds strictly concerned with gesture research. For example, the DG Infor- tion Society of the European Commission (www.cordis.lu/ist) supports several initiatives, such as the “Disappearing Computer” and “Presence” EU-IST FET (Future and Emerging Technologies), the IST program “Interfaces & Enhanced Audio-Visual Services” (see for example the project MEGA, Multisensory - pressive Gesture Applications, www.megaproject.org), and the IST strategic - jective “Multimodal Interfaces.” Several EC projects and other funded research are represented in the chapters of this book. Awiderangeofapplicationscanbene?tfromadvancesinresearchongesture, from consolidated areas such as surveillance to new or emerging ?elds such as therapy and rehabilitation, home consumer goods, entertainment, and aud- visual, cultural and artistic applications, just to mention only a few of them.

This book constitutes the thoroughly refereed post-proceedings of the 6th International Workshop on Gesture in Human-Computer Interaction and Simulation, GW 2005, held in May 2005. The 22 revised long papers and 14 revised short papers presented together with 2 invited lectures were carefully selected from numerous submissions during two rounds of reviewing and improvement. The papers are organized in topical sections on human perception and production of gesture, sign language representation, sign language recognition, vision-based gesture recognition, gesture analysis, gesture synthesis, gesture and music, and gesture interaction in multimodal systems.