Stochastic Optimization

This book addresses stochastic optimization procedures in a broad manner. The first part offers an overview of relevant optimization philosophies; the second deals with benchmark problems in depth, by applying a selection of optimization procedures. Written primarily with scientists and students from the physical and engineering sciences in mind, this book addresses a larger community of all who wish to learn about stochastic optimization techniques and how to use them.

This textbook provides an introduction to convex duality for optimization problems in Banach spaces, integration theory, and their application to stochastic programming problems in a static or dynamic setting. It introduces and analyses the main algorithms for stochastic programs, while the theoretical aspects are carefully dealt with. The reader is shown how these tools can be applied to various fields, including approximation theory, semidefinite and second-order cone programming and linear decision rules. This textbook is recommended for students, engineers and researchers who are willing to take a rigorous approach to the mathematics involved in the application of duality theory to optimization with uncertainty.

This book covers not only foundational materials but also the most recent progresses made during the past few years on the area of machine learning algorithms. In spite of the intensive research and development in this area, there does not exist a systematic treatment to introduce the fundamental concepts and recent progresses on machine learning algorithms, especially on those based on stochastic optimization methods, randomized algorithms, nonconvex optimization, distributed and online learning, and projection free methods. This book will benefit the broad audience in the area of machine learning, artificial intelligence and mathematical programming community by presenting these recent developments in a tutorial style, starting from the basic building blocks to the most carefully designed and complicated algorithms for machine learning.

This book examines optimization problems that in practice involve random model parameters. It details the computation of robust optimal solutions, i.e., optimal solutions that are insensitive with respect to random parameter variations, where appropriate deterministic substitute problems are needed. Based on the probability distribution of the random data and using decision theoretical concepts, optimization problems under stochastic uncertainty are converted into appropriate deterministic substitute problems. Due to the probabilities and expectations involved, the book also shows how to apply approximative solution techniques. Several deterministic and stochastic approximation methods are provided: Taylor expansion methods, regression and response surface methods (RSM), probability inequalities, multiple linearization of survival/failure domains, discretization methods, convex approximation/deterministic descent directions/efficient points, stochastic approximation and gradient procedures and differentiation formulas for probabilities and expectations. In the third edition, this book further develops stochastic optimization methods. In particular, it now shows how to apply stochastic optimization methods to the approximate solution of important concrete problems arising in engineering, economics and operations research.

Stochastic programming is the study of procedures for decision making under the presence of uncertainties and risks. Stochastic programming approaches have been successfully used in a number of areas such as energy and production planning, telecommunications, and transportation. Recently, the practical experience gained in stochastic programming has been expanded to a much larger spectrum of applications including financial modeling, risk management, and probabilistic risk analysis. Major topics in this volume include: (1) advances in theory and implementation of stochastic programming algorithms; (2) sensitivity analysis of stochastic systems; (3) stochastic programming applications and other related topics. Audience: Researchers and academies working in optimization, computer modeling, operations research and financial engineering. The book is appropriate as supplementary reading in courses on optimization and financial engineering.

A reprint of one of the classic volumes on portfolio theory and investment, this book has been used by the leading professors at universities such as Stanford, Berkeley, and Carnegie-Mellon. It contains five parts, each with a review of the literature and about 150 pages of computational and review exercises and further in-depth, challenging problems.Frequently referenced and highly usable, the material remains as fresh and relevant for a portfolio theory course as ever.

Multistage stochastic optimization problems appear in many ways in finance, insurance, energy production and trading, logistics and transportation, among other areas. They describe decision situations under uncertainty and with a longer planning horizon. This book contains a comprehensive treatment of today’s state of the art in multistage stochastic optimization. It covers the mathematical backgrounds of approximation theory as well as numerous practical algorithms and examples for the generation and handling of scenario trees. A special emphasis is put on estimation and bounding of the modeling error using novel distance concepts, on time consistency and the role of model ambiguity in the decision process. An extensive treatment of examples from electricity production, asset liability management and inventory control concludes the book.

This rapidly developing field encompasses many disciplines including operations research, mathematics, and probability. Conversely, it is being applied in a wide variety of subjects ranging from agriculture to financial planning and from industrial engineering to computer networks. This textbook provides a first course in stochastic programming suitable for students with a basic knowledge of linear programming, elementary analysis, and probability. The authors present a broad overview of the main themes and methods of the subject, thus helping students develop an intuition for how to model uncertainty into mathematical problems, what uncertainty changes bring to the decision process, and what techniques help to manage uncertainty in solving the problems. The early chapters introduce some worked examples of stochastic programming, demonstrate how a stochastic model is formally built, develop the properties of stochastic programs and the basic solution techniques used to solve them. The book then goes on to cover approximation and sampling techniques and is rounded off by an in-depth case study. A well-paced and wide-ranging introduction to this subject.