Multimedia Information Extraction

The advent of increasingly large consumer collections of audio (e.g., iTunes), imagery (e.g., Flickr), and video (e.g., YouTube) is driving a need not only for multimedia retrieval but also information extraction from and across media. Furthermore, industrial and government collections fuel requirements for stock media access, media preservation, broadcast news retrieval, identity management, and video surveillance. While significant advances have been made in language processing for information extraction from unstructured multilingual text and extraction of objects from imagery and video, these advances have been explored in largely independent research communities who have addressed extracting information from single media (e.g., text, imagery, audio). And yet users need to search for concepts across individual media, author multimedia artifacts, and perform multimedia analysis in many domains. This collection is intended to serve several purposes, including reporting the current state of the art, stimulating novel research, and encouraging cross-fertilization of distinct research disciplines. The collection and integration of a common base of intellectual material will provide an invaluable service from which to teach a future generation of cross disciplinary media scientists and engineers.

This book presents the state of the art in the areas of ontology evolution and knowledge-driven multimedia information extraction, placing an emphasis on how the two can be combined to bridge the semantic gap. This was also the goal of the EC-sponsored BOEMIE (Bootstrapping Ontology Evolution with Multimedia Information Extraction) project, to which the authors of this book have all contributed. The book addresses researchers and practitioners in the field of computer science and more specifically in knowledge representation and management, ontology evolution, and information extraction from multimedia data. It may also constitute an excellent guide to students attending courses within a computer science study program, addressing information processing and extraction from any type of media (text, images, and video). Among other things, the book gives concrete examples of how several of the methods discussed can be applied to athletics (track and field) events.

The volume focuses on research-oriented work, which can help opening up new vistas of research for the research community, and explore new mechanisms of retrieval of information from multimedia documents, particularly from heritage documents, apart from using the conventional methods.

Based on more than 10 years of teaching experience, Blanken and his coeditors have assembled all the topics that should be covered in advanced undergraduate or graduate courses on multimedia retrieval and multimedia databases. The single chapters of this textbook explain the general architecture of multimedia information retrieval systems and cover various metadata languages such as Dublin Core, RDF, or MPEG. The authors emphasize high-level features and show how these are used in mathematical models to support the retrieval process. For each chapter, there’s detail on further reading, and additional exercises and teaching material is available online.

At its very core multimedia information retrieval means the process of searching for and finding multimedia documents; the corresponding research field is concerned with building the best possible multimedia search engines. The intriguing bit here is that the query itself can be a multimedia excerpt: For example, when you walk around in an unknown place and stumble across an interesting landmark, would it not be great if you could just take a picture with your mobile phone and send it to a service that finds a similar picture in a database and tells you more about the building -- and about its significance, for that matter? This book goes further by examining the full matrix of a variety of query modes versus document types. How do you retrieve a music piece by humming? What if you want to find news video clips on forest fires using a still image? The text discusses underlying techniques and common approaches to facilitate multimedia search engines from metadata driven retrieval, via piggy-back text retrieval where automated processes create text surrogates for multimedia, automated image annotation and content-based retrieval. The latter is studied in great depth looking at features and distances, and how to effectively combine them for efficient retrieval, to a point where the readers have the ingredients and recipe in their hands for building their own multimedia search engines. Supporting users in their resource discovery mission when hunting for multimedia material is not a technological indexing problem alone. We look at interactive ways of engaging with repositories through browsing and relevance feedback, roping in geographical context, and providing visual summaries for videos. The book concludes with an overview of state-of-the-art research projects in the area of multimedia information retrieval, which gives an indication of the research and development trends and, thereby, a glimpse of the future world. Table of Contents: What is Multimedia Information Retrieval? / Basic Multimedia Search Technologies / Content-based Retrieval in Depth / Added Services / Multimedia Information Retrieval Research / Summary

Everything you ever wanted to know about multimedia retrieval and management. This comprehensive book offers a full picture of the cutting-edge technologies necessary for a profound introduction to the field. Leading experts also cover a broad range of practical applications.

Multimedia Information Retrieval: Content-Based Information Retrieval from Large Text and Audio Databases addresses the future need for sophisticated search techniques that will be required to find relevant information in large digital data repositories, such as digital libraries and other multimedia databases. Because of the dramatically increasing amount of multimedia data available, there is a growing need for new search techniques that provide not only fewer bits, but also the most relevant bits, to those searching for multimedia digital data. This book serves to bridge the gap between classic ranking of text documents and modern information retrieval where composite multimedia documents are searched for relevant information. Multimedia Information Retrieval: Content-Based Information Retrieval from Large Text and Audio Databases begins to pave the way for speech retrieval; only recently has the search for information in speech recordings become feasible. This book provides the necessary introduction to speech recognition while discussing probabilistic retrieval and text retrieval, key topics in classic information retrieval. The book then discusses speech retrieval, which is even more challenging than retrieving text documents because word boundaries are difficult to detect, and recognition errors affect the retrieval effectiveness. This book also addresses the problem of integrating information retrieval and database functions, since there is an increasing need for retrieving information from frequently changing data collections which are organized and managed by a database system. Multimedia Information Retrieval: Content-Based Information Retrieval from Large Text and Audio Databases serves as an excellent reference source and may be used as a text for advanced courses on the topic.

At its very core multimedia information retrieval means the process of searching for and finding multimedia documents; the corresponding research field is concerned with building the best possible multimedia search engines. The intriguing bit here is that the query itself can be a multimedia excerpt: For example, when you walk around in an unknown place and stumble across an interesting landmark, would it not be great if you could just take a picture with your mobile phone and send it to a service that finds a similar picture in a database and tells you more about the building -- and about its significance, for that matter? This book goes further by examining the full matrix of a variety of query modes versus document types. How do you retrieve a music piece by humming? What if you want to find news video clips on forest fires using a still image? The text discusses underlying techniques and common approaches to facilitate multimedia search engines from metadata driven retrieval, via piggy-back text retrieval where automated processes create text surrogates for multimedia, automated image annotation and content-based retrieval. The latter is studied in great depth looking at features and distances, and how to effectively combine them for efficient retrieval, to a point where the readers have the ingredients and recipe in their hands for building their own multimedia search engines. Supporting users in their resource discovery mission when hunting for multimedia material is not a technological indexing problem alone. We look at interactive ways of engaging with repositories through browsing and relevance feedback, roping in geographical context, and providing visual summaries for videos. The book concludes with an overview of state-of-the-art research projects in the area of multimedia information retrieval, which gives an indication of the research and development trends and, thereby, a glimpse of the future world. Table of Contents: What is Multimedia Information Retrieval? / Basic Multimedia Search Technologies / Content-based Retrieval in Depth / Added Services / Multimedia Information Retrieval Research / Summary