Evaluation Forum

This volume celebrates the twentieth anniversary of CLEF - the Cross-Language Evaluation Forum for the first ten years, and the Conference and Labs of the Evaluation Forum since – and traces its evolution over these first two decades. CLEF’s main mission is to promote research, innovation and development of information retrieval (IR) systems by anticipating trends in information management in order to stimulate advances in the field of IR system experimentation and evaluation. The book is divided into six parts. Parts I and II provide background and context, with the first part explaining what is meant by experimental evaluation and the underlying theory, and describing how this has been interpreted in CLEF and in other internationally recognized evaluation initiatives. Part II presents research architectures and infrastructures that have been developed to manage experimental data and to provide evaluation services in CLEF and elsewhere. Parts III, IV and V represent the core of the book, presenting some of the most significant evaluation activities in CLEF, ranging from the early multilingual text processing exercises to the later, more sophisticated experiments on multimodal collections in diverse genres and media. In all cases, the focus is not only on describing “what has been achieved”, but above all on “what has been learnt”. The final part examines the impact CLEF has had on the research world and discusses current and future challenges, both academic and industrial, including the relevance of IR benchmarking in industrial settings. Mainly intended for researchers in academia and industry, it also offers useful insights and tips for practitioners in industry working on the evaluation and performance issues of IR tools, and graduate students specializing in information retrieval.

Potential health effects from chemicals that disrupt endocrine function pose an environmental health concern because of their ability to interfere with normal hormone function in human and wildlife populations. The endocrine system regulates biological processes throughout the body and is sensitive to small changes in hormone concentrations. Endocrine-disruptor research has focused primarily on chemicals that affect three hormone pathways that play important roles in reproduction and development - the estrogen, androgen, and thyroid hormone pathways. Some of this research has identified dose-response relationships that have nonmonotonic curves. Nonmonotonic dose-response curves (NMDRs) are of concern because they do not follow the usual assumption made in toxicology that as dose decreases the response also decreases. The existence of NMDRs has been a controversial topic for decades, and there has been considerable debate about their implications for how chemicals are tested and for how risks from such chemicals are assessed. Toxicity tests are designed to identify hazards and to characterize dose-response relationships, so tests are aimed at finding a (high) dose that elicits a response, and dose-response is explored by testing lower doses spaced to identify statistically a no- or lowest-observed-adverse-effect level. The concern for NMDRs is that such studies, as currently designed, might not detect the inflection of the dose-response curve if only a few doses are tested or if the change in inflection occurs below the range of doses tested. Another concern is that some NMDRs are found for biological effects that are not usually evaluated in toxicity tests. If current testing strategies are inadequate to account for NMDRs, changes to risk assessment practices might be necessary. To help address these issues, the U.S. Environmental Protection Agency (EPA) developed a draft State-of-the-Science Evaluation: Nonmonotonic Dose Responses as they Apply to Estrogen, Androgen, and Thyroid Pathways and EPA Testing and Assessment Procedures. EPA asked the National Research Council to conduct an independent review of this evaluation to ensure that it is scientifically sound and of high quality. Review of Environmental Protection Agency's State-of-the-Science Evaluation of Nonmonotonic Dose-Response as they Apply to Endocrine Disrupters evaluates whether EPA's evaluation presents a scientifically sound and high-quality analysis of the literature on NMDRs. This report reviews how well the EPA evaluation described how the assessment was performed, whether consistent methods and criteria were applied in the analysis of different evidence streams, and whether appropriate methods were applied to evaluating the evidence. The report makes recommendations to improve EPA's process and strengthen the evaluation.

This book constitutes the thoroughly refereed postproceedings of the 4th Workshop of the Cross-Language Evaluation Forum, CLEF 2003, held in Trondheim, Norway in August 2003. The 61 revised papers presented together with an introduction were carefully reviewed and selected for inclusion in the book. The papers are organized in topical sections on mainly cross-language experiments, mono lingual experiments, domain-specific document retrieval, interactive cross-language retrieval, cross-language question answering, cross-language image retrieval, and cross-language spoken document retrieval.

The Encyclopedia of Evaluation is an authoritative, first-of-its-kind who, what, where, why, and how of the field of evaluation. Covering professional practice as well as academia, this volume chronicles the development of the field—its history, key figures, theories, approaches, and goals. From the leading publisher in the field of evaluation, this work is a must-have for all social science libraries, departments that offer courses in evaluation, and students and professional evaluators around the world. The entries in this Encyclopedia capture the essence of evaluation as a practice (methods, techniques, roles, people), as a profession (professional obligations, shared knowledge, ethical imperatives, events, places) and as a discipline (theories and models of evaluation, ontological and epistemological issues).

This book constitutes the thoroughly refereed proceedings of the 10th Workshop of the Cross Language Evaluation Forum, CLEF 2010, held in Corfu, Greece, in September/October 2009. The volume reports experiments on various types of multimedia collections. It is divided into three main sections presenting the results of the following tracks: Interactive Cross-Language Retrieval (iCLEF), Cross-Language Image Retrieval (ImageCLEF), and Cross-Language Video Retrieval (VideoCLEF).