Experimental Methods For The Analysis Of Optimization Algorithms

In operations research and computer science it is common practice to evaluate the performance of optimization algorithms on the basis of computational results, and the experimental approach should follow accepted principles that guarantee the reliability and reproducibility of results. However, computational experiments differ from those in other sciences, and the last decade has seen considerable methodological research devoted to understanding the particular features of such experiments and assessing the related statistical methods. This book consists of methodological contributions on different scenarios of experimental analysis. The first part overviews the main issues in the experimental analysis of algorithms, and discusses the experimental cycle of algorithm development; the second part treats the characterization by means of statistical distributions of algorithm performance in terms of solution quality, runtime and other measures; and the third part collects advanced methods from experimental design for configuring and tuning algorithms on a specific class of instances with the goal of using the least amount of experimentation. The contributor list includes leading scientists in algorithm design, statistical design, optimization and heuristics, and most chapters provide theoretical background and are enriched with case studies. This book is written for researchers and practitioners in operations research and computer science who wish to improve the experimental assessment of optimization algorithms and, consequently, their design.

This book offers the first comprehensive taxonomy for multimodal optimization algorithms, work with its root in topics such as niching, parallel evolutionary algorithms, and global optimization. The author explains niching in evolutionary algorithms and its benefits; he examines their suitability for use as diagnostic tools for experimental analysis, especially for detecting problem (type) properties; and he measures and compares the performances of niching and canonical EAs using different benchmark test problem sets. His work consolidates the recent successes in this domain, presenting and explaining use cases, algorithms, and performance measures, with a focus throughout on the goals of the optimization processes and a deep understanding of the algorithms used. The book will be useful for researchers and practitioners in the area of computational intelligence, particularly those engaged with heuristic search, multimodal optimization, evolutionary computing, and experimental analysis.

This book introduces the new experimentalism in evolutionary computation, providing tools to understand algorithms and programs and their interaction with optimization problems. It develops and applies statistical techniques to analyze and compare modern search heuristics such as evolutionary algorithms and particle swarm optimization. The book bridges the gap between theory and experiment by providing a self-contained experimental methodology and many examples.

Metaheuristics, and evolutionary algorithms in particular, are known to provide efficient, adaptable solutions for many real-world problems, but the often informal way in which they are defined and applied has led to misconceptions, and even successful applications are sometimes the outcome of trial and error. Ideally, theoretical studies should explain when and why metaheuristics work, but the challenge is huge: mathematical analysis requires significant effort even for simple scenarios and real-life problems are usually quite complex. In this book the editors establish a bridge between theory and practice, presenting principled methods that incorporate problem knowledge in evolutionary algorithms and other metaheuristics. The book consists of 11 chapters dealing with the following topics: theoretical results that show what is not possible, an assessment of unsuccessful lines of empirical research; methods for rigorously defining the appropriate scope of problems while acknowledging the compromise between the class of problems to which a search algorithm is applied and its overall expected performance; the top-down principled design of search algorithms, in particular showing that it is possible to design algorithms that are provably good for some rigorously defined classes; and, finally, principled practice, that is reasoned and systematic approaches to setting up experiments, metaheuristic adaptation to specific problems, and setting parameters. With contributions by some of the leading researchers in this domain, this book will be of significant value to scientists, practitioners, and graduate students in the areas of evolutionary computing, metaheuristics, and computational intelligence.

​This book aims at illustrating strategies to account for uncertainty in complex systems described by computer simulations. When optimizing the performances of these systems, accounting or neglecting uncertainty may lead to completely different results; therefore, uncertainty management is a major issues in simulation-optimization. Because of its wide field of applications, simulation-optimization issues have been addressed by different communities with different methods, and from slightly different perspectives. Alternative approaches have been developed, also depending on the application context, without any well-established method clearly outperforming the others. This editorial project brings together — as chapter contributors — researchers from different (though interrelated) areas; namely, statistical methods, experimental design, stochastic programming, global optimization, metamodeling, and design and analysis of computer simulation experiments. Editors’ goal is to take advantage of such a multidisciplinary environment, to offer to the readers a much deeper understanding of the commonalities and differences of the various approaches to simulation-based optimization, especially in uncertain environments. Editors aim to offer a bibliographic reference on the topic, enabling interested readers to learn about the state-of-the-art in this research area, also accounting for potential real-world applications to improve also the state-of-the-practice. Besides researchers and scientists of the field, the primary audience for the proposed book includes PhD students, academic teachers, as well as practitioners and professionals. Each of these categories of potential readers present adequate channels for marketing actions, e.g. scientific, academic or professional societies, internet-based communities, and authors or buyers of related publications.​

This book constitutes the refereed post-conference proceedings of the Special Event on the Analysis of Experimental Algorithms, SEA2 2019, held in Kalamata, Greece, in June 2019. The 35 revised full papers presented were carefully reviewed and selected from 45 submissions. The papers cover a wide range of topics in both computer science and operations research/mathematical programming. They focus on the role of experimentation and engineering techniques in the design and evaluation of algorithms, data structures, and computational optimization methods.

This edited volume illustrates the connections between machine learning techniques, black box optimization, and no-free lunch theorems. Each of the thirteen contributions focuses on the commonality and interdisciplinary concepts as well as the fundamentals needed to fully comprehend the impact of individual applications and problems. Current theoretical, algorithmic, and practical methods used are provided to stimulate a new effort towards innovative and efficient solutions. The book is intended for beginners who wish to achieve a broad overview of optimization methods and also for more experienced researchers as well as researchers in mathematics, optimization, operations research, quantitative logistics, data analysis, and statistics, who will benefit from access to a quick reference to key topics and methods. The coverage ranges from mathematically rigorous methods to heuristic and evolutionary approaches in an attempt to equip the reader with different viewpoints of the same problem.

This book constitutes the refereed proceedings of the 7th International Workshop on Experimental and Efficient Algorithms, WEA 2008, held in Provincetown, MA, USA, in May/June 2008. The 26 revised full papers were carefully reviewed and selected from numerous submissions and present current research on experimental evaluation and engineering of algorithms, as well as in various aspects of computational optimization and its applications. Special focus is put on the use of experimental methods to guide the design, analysis, implementation, and evaluation of algorithms, heuristics, and optimization programs.