Optimization By The Pattern Search Method

An infinite number of mathematical models can be created to describe naturally occurring physical systems. In addition there is a smaller, but large, number of criteria which can be used to measure the goodness of fit of a model to a physical process. Together, a selected criterion and a mathematical model define an objective function to be optimized.

The book is intended for graduate students and researchers in mathematics, computer science, and operational research. The book presents a new derivative-free optimization method/algorithm based on randomly generated trial points in specified domains and where the best ones are selected at each iteration by using a number of rules. This method is different from many other well established methods presented in the literature and proves to be competitive for solving many unconstrained optimization problems with different structures and complexities, with a relative large number of variables. Intensive numerical experiments with 140 unconstrained optimization problems, with up to 500 variables, have shown that this approach is efficient and robust. Structured into 4 chapters, Chapter 1 is introductory. Chapter 2 is dedicated to presenting a two level derivative-free random search method for unconstrained optimization. It is assumed that the minimizing function is continuous, lower bounded and its minimum value is known. Chapter 3 proves the convergence of the algorithm. In Chapter 4, the numerical performances of the algorithm are shown for solving 140 unconstrained optimization problems, out of which 16 are real applications. This shows that the optimization process has two phases: the reduction phase and the stalling one. Finally, the performances of the algorithm for solving a number of 30 large-scale unconstrained optimization problems up to 500 variables are presented. These numerical results show that this approach based on the two level random search method for unconstrained optimization is able to solve a large diversity of problems with different structures and complexities. There are a number of open problems which refer to the following aspects: the selection of the number of trial or the number of the local trial points, the selection of the bounds of the domains where the trial points and the local trial points are randomly generated and a criterion for initiating the line search.

Computational optimization is an important paradigm with a wide range of applications. In virtually all branches of engineering and industry, we almost always try to optimize something - whether to minimize the cost and energy consumption, or to maximize profits, outputs, performance and efficiency. In many cases, this search for optimality is challenging, either because of the high computational cost of evaluating objectives and constraints, or because of the nonlinearity, multimodality, discontinuity and uncertainty of the problem functions in the real-world systems. Another complication is that most problems are often NP-hard, that is, the solution time for finding the optimum increases exponentially with the problem size. The development of efficient algorithms and specialized techniques that address these difficulties is of primary importance for contemporary engineering, science and industry. This book consists of 12 self-contained chapters, contributed from worldwide experts who are working in these exciting areas. The book strives to review and discuss the latest developments concerning optimization and modelling with a focus on methods and algorithms for computational optimization. It also covers well-chosen, real-world applications in science, engineering and industry. Main topics include derivative-free optimization, multi-objective evolutionary algorithms, surrogate-based methods, maximum simulated likelihood estimation, support vector machines, and metaheuristic algorithms. Application case studies include aerodynamic shape optimization, microwave engineering, black-box optimization, classification, economics, inventory optimization and structural optimization. This graduate level book can serve as an excellent reference for lecturers, researchers and students in computational science, engineering and industry.

During the last decade the techniques of non-linear optim ization have emerged as an important subject for study and research. The increasingly widespread application of optim ization has been stimulated by the availability of digital computers, and the necessity of using them in the investigation of large systems. This book is an introduction to non-linear methods of optimization and is suitable for undergraduate and post graduate courses in mathematics, the physical and social sciences, and engineering. The first half of the book covers the basic optimization techniques including linear search methods, steepest descent, least squares, and the Newton-Raphson method. These are described in detail, with worked numerical examples, since they form the basis from which advanced methods are derived. Since 1965 advanced methods of unconstrained and constrained optimization have been developed to utilise the computational power of the digital computer. The second half of the book describes fully important algorithms in current use such as variable metric methods for unconstrained problems and penalty function methods for constrained problems. Recent work, much of which has not yet been widely applied, is reviewed and compared with currently popular techniques under a few generic main headings. vi PREFACE Chapter I describes the optimization problem in mathemat ical form and defines the terminology used in the remainder of the book. Chapter 2 is concerned with single variable optimization. The main algorithms of both search and approximation methods are developed in detail since they are an essential part of many multi-variable methods.

Primarily designed as a text for the postgraduate students of mechanical engineering and related branches, it provides an excellent introduction to optimization methods—the overview, the history, and the development. It is equally suitable for the undergraduate students for their electives. The text then moves on to familiarize the students with the formulation of optimization problems, graphical solutions, analytical methods of nonlinear optimization, classical optimization techniques, single variable (one-dimensional) unconstrained optimization, multidimensional problems, constrained optimization, equality and inequality constraints. With complexities of human life, the importance of optimization techniques as a tool has increased manifold. The application of optimization techniques creates an efficient, effective and a better life. Features • Includes numerous illustrations and unsolved problems. • Contains university questions. • Discusses the topics with step-by-step procedures.

This textbook explores reactive power control and voltage stability and explains how they relate to different forms of power generation and transmission. Bringing together international experts in this field, it includes chapters on electric power analysis, design and operational strategies. The book explains fundamental concepts before moving on to report on the latest theoretical findings in reactive power control, including case studies and advice on practical implementation students can use to design their own research projects. Featuring numerous worked-out examples, problems and solutions, as well as over 400 illustrations, Reactive Power Control in AC Power Systems offers an essential textbook for postgraduate students in electrical power engineering. It offers practical advice on implementing the methods discussed in the book using MATLAB and DIgSILENT, and the relevant program files are available at extras.springer.com.

For many engineering problems we require optimization processes with dynamic adaptation as we aim to establish the dimension of the search space where the optimum solution resides and develop robust techniques to avoid the local optima usually associated with multimodal problems. This book explores multidimensional particle swarm optimization, a technique developed by the authors that addresses these requirements in a well-defined algorithmic approach. After an introduction to the key optimization techniques, the authors introduce their unified framework and demonstrate its advantages in challenging application domains, focusing on the state of the art of multidimensional extensions such as global convergence in particle swarm optimization, dynamic data clustering, evolutionary neural networks, biomedical applications and personalized ECG classification, content-based image classification and retrieval, and evolutionary feature synthesis. The content is characterized by strong practical considerations, and the book is supported with fully documented source code for all applications presented, as well as many sample datasets. The book will be of benefit to researchers and practitioners working in the areas of machine intelligence, signal processing, pattern recognition, and data mining, or using principles from these areas in their application domains. It may also be used as a reference text for graduate courses on swarm optimization, data clustering and classification, content-based multimedia search, and biomedical signal processing applications.