Database Explorations

A note from the authors: Dear Reader: "Database is boring." That sentiment is heard all too widely these days. But it's so wrong! The database field is full of important problems still to be solved and interesting issues still to be examined - and some of those problems and issues are explored in this book. Between us, we have nearly 80 years experience in this field, and we're still actively researching, exploring, and learning, as well as helping others do the same. The present book is the latest in a series devoted to these goals; using "The Third Manifesto" (a detailed proposal for the future of database technology) as a foundation, it reports on some of our most recent investigations in this field. Among many other things, it includes the most recent version of "The Third Manifesto" itself; specifications for a conforming language called Tutorial D; and a detailed proposal for a model of type inheritance. Other significant features include: - Extending the foreign key concept - Simplifying queries using image relations - Closer looks at logic and relational algebra - Suggested approaches to "missing information" - Responses to certain "Manifesto" criticisms - Clarifying aspects of normalization The tone of the book overall is naturally somewhat serious, but there are moments of light relief as well. We hope you enjoy it. C.J. Date and Hugh Darwen

Fuzzy Modeling and Genetic Algorithms for Data Mining and Exploration is a handbook for analysts, engineers, and managers involved in developing data mining models in business and government. As you’ll discover, fuzzy systems are extraordinarily valuable tools for representing and manipulating all kinds of data, and genetic algorithms and evolutionary programming techniques drawn from biology provide the most effective means for designing and tuning these systems. You don’t need a background in fuzzy modeling or genetic algorithms to benefit, for this book provides it, along with detailed instruction in methods that you can immediately put to work in your own projects. The author provides many diverse examples and also an extended example in which evolutionary strategies are used to create a complex scheduling system. Written to provide analysts, engineers, and managers with the background and specific instruction needed to develop and implement more effective data mining systems Helps you to understand the trade-offs implicit in various models and model architectures Provides extensive coverage of fuzzy SQL querying, fuzzy clustering, and fuzzy rule induction Lays out a roadmap for exploring data, selecting model system measures, organizing adaptive feedback loops, selecting a model configuration, implementing a working model, and validating the final model In an extended example, applies evolutionary programming techniques to solve a complicated scheduling problem Presents examples in C, C++, Java, and easy-to-understand pseudo-code Extensive online component, including sample code and a complete data mining workbench

This text integrates different mobility data handling processes, from database management to multi-dimensional analysis and mining, into a unified presentation driven by the spectrum of requirements raised by real-world applications. It presents a step-by-step methodology to understand and exploit mobility data: collecting and cleansing data, storage in Moving Object Database (MOD) engines, indexing, processing, analyzing and mining mobility data. Emerging issues, such as semantic and privacy-aware querying and mining as well as distributed data processing, are also covered. Theoretical presentation is smoothly interchanged with hands-on exercises and case studies involving an actual MOD engine. The authors are established experts who address both theoretical and practical dimensions of the field but also present valuable prototype software. The background context, clear explanations and sample exercises make this an ideal textbook for graduate students studying database management, data mining and geographic information systems.

Data usually comes in a plethora of formats and dimensions, rendering the exploration and information extraction processes challenging. Thus, being able to perform exploratory analyses in the data with the intent of having an immediate glimpse on some of the data properties is becoming crucial. Exploratory analyses should be simple enough to avoid complicate declarative languages (such as SQL) and mechanisms, and at the same time retain the flexibility and expressiveness of such languages. Recently, we have witnessed a rediscovery of the so-called example-based methods, in which the user, or the analyst, circumvents query languages by using examples as input. An example is a representative of the intended results, or in other words, an item from the result set. Example-based methods exploit inherent characteristics of the data to infer the results that the user has in mind, but may not able to (easily) express. They can be useful in cases where a user is looking for information in an unfamiliar dataset, when the task is particularly challenging like finding duplicate items, or simply when they are exploring the data. In this book, we present an excursus over the main methods for exploratory analysis, with a particular focus on example-based methods. We show how that different data types require different techniques, and present algorithms that are specifically designed for relational, textual, and graph data. The book presents also the challenges and the new frontiers of machine learning in online settings which recently attracted the attention of the database community. The lecture concludes with a vision for further research and applications in this area.

Helps readers to transition from traditional statistics to modern data science Reviews the pros and cons of open source and commercial software packages, and their proper applications in specific situations. Explores data using dynamic methods rather than counting on dichotomous thinking. Considers alternate models using ensemble models and model comparison rather than fixing a preconceived hypothesis/model on a single method. Shows how to find the hidden pattern in the data by dynamic visualization rather than over-relying on numeric results.

"More than 150 topic experts, data scientists, civic ""hackers,"" civil society groups and development practitioners gathered at the World Bank on March 15-17, 2013 for the DC Big Data Exploration event. Working alongside Bank experts on the Poverty and Fraud & Corruption teams, data scientists uncovered new ways of collecting, exploring, and visualizing data to maximize their impact. The collaboration between the two communities yielded new insights from World Bank data, devised new ways of using existing big data sources for monitoring poverty and corruption, and created entirely new streams of data that the Bank and its partners can use in future research. This comprehensive final report on the DC Big Data Exploration provides an in-depth look at several ‘big’ questions and includes recommendations and next steps for maximizing impact and outcomes from the initial hypotheses. "

Exploration of Visual Data presents latest research efforts in the area of content-based exploration of image and video data. The main objective is to bridge the semantic gap between high-level concepts in the human mind and low-level features extractable by the machines. The two key issues emphasized are "content-awareness" and "user-in-the-loop". The authors provide a comprehensive review on algorithms for visual feature extraction based on color, texture, shape, and structure, and techniques for incorporating such information to aid browsing, exploration, search, and streaming of image and video data. They also discuss issues related to the mixed use of textual and low-level visual features to facilitate more effective access of multimedia data. Exploration of Visual Data provides state-of-the-art materials on the topics of content-based description of visual data, content-based low-bitrate video streaming, and latest asymmetric and nonlinear relevance feedback algorithms, which to date are unpublished.

A cutting-edge guide to the analysis of DNA microarray data Genomics is one of the major scientific revolutions of this century, and the use of microarrays to rapidly analyze numerous DNA samples has enabled scientists to make sense of mountains of genomic data through statistical analysis. Today, microarrays are being used in biomedical research to study such vital areas as a drug’s therapeutic value–or toxicity–and cancer-spreading patterns of gene activity. Exploration and Analysis of DNA Microarray and Protein Array Data answers the need for a comprehensive, cutting-edge overview of this important and emerging field. The authors, seasoned researchers with extensive experience in both industry and academia, effectively outline all phases of this revolutionary analytical technique, from the preprocessing to the analysis stage. Highlights of the text include: A review of basic molecular biology, followed by an introduction to microarrays and their preparation Chapters on processing scanned images and preprocessing microarray data Methods for identifying differentially expressed genes in comparative microarray experiments Discussions of gene and sample clustering and class prediction Extension of analysis methods to protein array data Numerous exercises for self-study as well as data sets and a useful collection of computational tools on the authors’ Web site make this important text a valuable resource for both students and professionals in the field.