The Technology Of Binaural Understanding

Sound, devoid of meaning, would not matter to us. It is the information sound conveys that helps the brain to understand its environment. Sound and its underlying meaning are always associated with time and space. There is no sound without spatial properties, and the brain always organizes this information within a temporal–spatial framework. This book is devoted to understanding the importance of meaning for spatial and related further aspects of hearing, including cross-modal inference. People, when exposed to acoustic stimuli, do not react directly to what they hear but rather to what they hear means to them. This semiotic maxim may not always apply, for instance, when the reactions are reflexive. But, where it does apply, it poses a major challenge to the builders of models of the auditory system. Take, for example, an auditory model that is meant to be implemented on a robotic agent for autonomous search-&-rescue actions. Or think of a system that can perform judgments on the sound quality of multimedia-reproduction systems. It becomes immediately clear that such a system needs • Cognitive capabilities, including substantial inherent knowledge • The ability to integrate information across different sensory modalities To realize these functions, the auditory system provides a pair of sensory organs, the two ears, and the means to perform adequate preprocessing of the signals provided by the ears. This is realized in the subcortical parts of the auditory system. In the title of a prior book, the term Binaural Listening is used to indicate a focus on sub-cortical functions. Psychoacoustics and auditory signal processing contribute substantially to this area. The preprocessed signals are then forwarded to the cortical parts of the auditory system where, among other things, recognition, classification, localization, scene analysis, assignment of meaning, quality assessment, and action planning take place. Also, information from different sensory modalities is integrated at this level. Between sub-cortical and cortical regions of the auditory system, numerous feedback loops exist that ultimately support the high complexity and plasticity of the auditory system. The current book concentrates on these cognitive functions. Instead of processing signals, processing symbols is now the predominant modeling task. Substantial contributions to the field draw upon the knowledge acquired by cognitive psychology. The keyword Binaural Understanding in the book title characterizes this shift. Both books, The Technology of Binaural Listening and the current one, have been stimulated and supported by AABBA, an open research group devoted to the development and application of models of binaural hearing. The current book is dedicated to technologies that help explain, facilitate, apply, and support various aspects of binaural understanding. It is organized into five parts, each containing three to six chapters in order to provide a comprehensive overview of this emerging area. Each chapter was thoroughly reviewed by at least two anonymous, external experts. The first part deals with the psychophysical and physiological effects of Forming and Interpreting Aural Objects as well as the underlying models. The fundamental concepts of reflexive and reflective auditory feedback are introduced. Mechanisms of binaural attention and attention switching are covered—as well as how auditory Gestalt rules facilitate binaural understanding. A general blackboard architecture is introduced as an example of how machines can learn to form and interpret aural objects to simulate human cognitive listening. The second part, Configuring and Understanding Aural Space, focuses on the human understanding of complex three-dimensional environments—covering the psychological and biological fundamentals of auditory space formation. This part further addresses the human mechanisms used to process information and interact in complex reverberant environments, such as concert halls and forests, and additionally examines how the auditory system can learn to understand and adapt to these environments. The third part is dedicated to Processing Cross-Modal Inference and highlights the fundamental human mechanisms used to integrate auditory cues with cues from other modalities to localize and form perceptual objects. This part also provides a general framework for understanding how complex multimodal scenes can be simulated and rendered. The fourth part, Evaluating Aural-scene Quality and Speech Understanding, focuses on the object-forming aspects of binaural listening and understanding. It addresses cognitive mechanisms involved in both the understanding of speech and the processing of nonverbal information such as Sound Quality and Quality-of- Experience. The aesthetic judgment of rooms is also discussed in this context. Models that simulate underlying human processes and performance are covered in addition to techniques for rendering virtual environments that can then be used to test these models. The fifth part deals with the Application of Cognitive Mechanisms to Audio Technology. It highlights how cognitive mechanisms can be utilized to create spatial auditory illusions using binaural and other 3D-audio technologies. Further, it covers how cognitive binaural technologies can be applied to improve human performance in auditory displays and to develop new auditory technologies for interactive robots. The book concludes with the application of cognitive binaural technologies to the next generation of hearing aids.

This book reports on the application of advanced models of the human binaural hearing system in modern technology, among others, in the following areas: binaural analysis of aural scenes, binaural de-reverberation, binaural quality assessment of audio channels, loudspeakers and performance spaces, binaural perceptual coding, binaural processing in hearing aids and cochlea implants, binaural systems in robots, binaural/tactile human-machine interfaces, speech-intelligibility prediction in rooms and/or multi-speaker scenarios. An introduction to binaural modeling and an outlook to the future are provided. Further, the book features a MATLAB toolbox to enable readers to construct their own dedicated binaural models on demand.

The field of spatial hearing has exploded in the decade or so since Jens Blauert's classic work on acoustics was first published in English. This revised edition adds a new chapter that describes developments in such areas as auditory virtual reality (an important field of application that is based mainly on the physics of spatial hearing), binaural technology (modeling speech enhancement by binaural hearing), and spatial sound-field mapping. The chapter also includes recent research on the precedence effect that provides clear experimental evidence that cognition plays a significant role in spatial hearing.The remaining four chapters in this comprehensive reference cover auditory research procedures and psychometric methods, spatial hearing with one sound source, spatial hearing with multiple sound sources and in enclosed spaces, and progress and trends from 1972 (the first German edition) to 1983 (the first English edition) -- work that includes research on the physics of the external ear, and the application of signal processing theory to modeling the spatial hearing process. There is an extensive bibliography of more than 900 items.

The field of Binaural Hearing involves studies of auditory perception, physiology, and modeling, including normal and abnormal aspects of the system. Binaural processes involved in both sound localization and speech unmasking have gained a broader interest and have received growing attention in the published literature. The field has undergone some significant changes. There is now a much richer understanding of the many aspects that comprising binaural processing, its role in development, and in success and limitations of hearing-aid and cochlear-implant users. The goal of this volume is to provide an up-to-date reference on the developments and novel ideas in the field of binaural hearing. The primary readership for the volume is expected to be academic specialists in the diverse fields that connect with psychoacoustics, neuroscience, engineering, psychology, audiology, and cochlear implants. This volume will serve as an important resource by way of introduction to the field, in particular for graduate students, postdoctoral scholars, the faculty who train them and clinicians.

This book presents a comprehensive review of how acoustic waves are processed by the auditory system into structured sounds such as musical melodies, speech utterances, or environmental sounds. After an introduction, an overview is given of how the ears distribute acoustic information over a large array of frequency channels that contain the auditory information used by the central nervous system to generate a mental image of what is happening around the listener. This process, called auditory scene analysis, consists of two stages. In the first stage, auditory units are formed such as musical tones and speech syllables. Each auditory unit is perceived at a well-defined moment in time, the beat location of that auditory unit. Moreover, from this process of auditory-unit formation, the auditory attributes of these auditory units emerge, such as their timbre, their pitch, their loudness, and their perceived location. Each of these attributes is discussed in the corresponding chapter. In the second stage of auditory scene analysis, auditory-stream formation, the successive auditory units are integrated into auditory streams, i.e., temporally structured sequences of auditory units that are perceived as emanating from one and the same sound source. Examples of such auditory streams are musical melodies and the utterances of one speaker. The temporal structure of an auditory stream, its rhythm, is determined by the beat locations of its auditory units. The role played by the auditory attributes of the consecutive auditory units is discussed. The melodies of musical streams and the intonation contours of spoken utterances emerge from this process. In music, the beats of parallel streams generally fit into a metric pattern, and, depending on harmony, simultaneous tones can be perceived as consonant or dissonant. Finally, the book contains many sound examples including the MATLAB scripts with which they are generated.

In this work the importance of individualization in binaural technique is investigated. The results extend the present knowledge on the efficient measurement of individual head-related transfer functions (HRTFs) and highlight the importance of individual equalization filters in binaural reproduction, using both loudspeakers and headphones. Moreover, an integrated framework for the calculation of such equalization filters is presented. An innovative measurement setup was developed to allow the fast acquisition of individual HRTFs. The hardware was designed to be compatible with the range extrapolation technique. An individual HRTF dataset with 4000 directions can be measured in less than 6 minutes with this new setup. A framework was presented that integrates causality constraints to the regularized frequency domain calculation of crosstalk cancellation (CTC) filter. This framework also addresses the switching of active loudspeakers applying a weighted filter calculation method. A sound localization test showed that individualized CTC systems provide performance similar to that of binaural listening while nonindividualized CTC systems provide a significantly lower localization performance. Finally, a robust individual headphone equalization method was proposed. Perceptual tests showed that, in all but one of the tested situations, no audible differences between the original sound source and its binaural auditory display could be perceived.