Advanced Simulation Based Methods For Optimal Stopping And Control

This is an advanced guide to optimal stopping and control, focusing on advanced Monte Carlo simulation and its application to finance. Written for quantitative finance practitioners and researchers in academia, the book looks at the classical simulation based algorithms before introducing some of the new, cutting edge approaches under development.

Combining control theory and modeling, this textbook introduces and builds on methods for simulating and tackling concrete problems in a variety of applied sciences. Emphasizing "learning by doing," the authors focus on examples and applications to real-world problems. An elementary presentation of advanced concepts, proofs to introduce new ideas, and carefully presented MATLAB® programs help foster an understanding of the basics, but also lead the way to new, independent research. With minimal prerequisites and exercises in each chapter, this work serves as an excellent textbook and reference for graduate and advanced undergraduate students, researchers, and practitioners in mathematics, physics, engineering, computer science, as well as biology, biotechnology, economics, and finance.

This unique book presents real world success stories of collaboration between mathematicians and industrial partners, showcasing first-hand case studies, and lessons learned from the experiences, technologies, and business challenges that led to the successful development of industrial solutions based on mathematics. It shows the crucial contribution of mathematics to innovation and to the industrial creation of value, and the key position of mathematics in the handling of complex systems, amplifying innovation. Each story describes the challenge that led to the industrial cooperation, how the challenge was approached and how the solutions were achieved and implemented. When brought together, they illustrate the versatile European landscape of projects in almost all areas of applied mathematics and across all business sectors. This book of success stories has its origin in the Forward Look about Mathematics and Industry that was funded by the European Science Foundation (ESF) and coordinated by the Applied Mathematics Committee of the European Mathematical Society (EMS). In each of these success stories, researchers, students, entrepreneurs, policy makers and business leaders in a range of disciplines will find valuable material and important lessons that can be applied in their own fields.​

Developed from the author’s course at the Ecole Polytechnique, Monte-Carlo Methods and Stochastic Processes: From Linear to Non-Linear focuses on the simulation of stochastic processes in continuous time and their link with partial differential equations (PDEs). It covers linear and nonlinear problems in biology, finance, geophysics, mechanics, chemistry, and other application areas. The text also thoroughly develops the problem of numerical integration and computation of expectation by the Monte-Carlo method. The book begins with a history of Monte-Carlo methods and an overview of three typical Monte-Carlo problems: numerical integration and computation of expectation, simulation of complex distributions, and stochastic optimization. The remainder of the text is organized in three parts of progressive difficulty. The first part presents basic tools for stochastic simulation and analysis of algorithm convergence. The second part describes Monte-Carlo methods for the simulation of stochastic differential equations. The final part discusses the simulation of non-linear dynamics.

Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning introduce the evolving area of static and dynamic simulation-based optimization. Covered in detail are model-free optimization techniques – especially designed for those discrete-event, stochastic systems which can be simulated but whose analytical models are difficult to find in closed mathematical forms. Key features of this revised and improved Second Edition include: · Extensive coverage, via step-by-step recipes, of powerful new algorithms for static simulation optimization, including simultaneous perturbation, backtracking adaptive search and nested partitions, in addition to traditional methods, such as response surfaces, Nelder-Mead search and meta-heuristics (simulated annealing, tabu search, and genetic algorithms) · Detailed coverage of the Bellman equation framework for Markov Decision Processes (MDPs), along with dynamic programming (value and policy iteration) for discounted, average, and total reward performance metrics · An in-depth consideration of dynamic simulation optimization via temporal differences and Reinforcement Learning: Q-Learning, SARSA, and R-SMART algorithms, and policy search, via API, Q-P-Learning, actor-critics, and learning automata · A special examination of neural-network-based function approximation for Reinforcement Learning, semi-Markov decision processes (SMDPs), finite-horizon problems, two time scales, case studies for industrial tasks, computer codes (placed online) and convergence proofs, via Banach fixed point theory and Ordinary Differential Equations Themed around three areas in separate sets of chapters – Static Simulation Optimization, Reinforcement Learning and Convergence Analysis – this book is written for researchers and students in the fields of engineering (industrial, systems, electrical and computer), operations research, computer science and applied mathematics.

From the reviews: "Paul Glasserman has written an astonishingly good book that bridges financial engineering and the Monte Carlo method. The book will appeal to graduate students, researchers, and most of all, practicing financial engineers [...] So often, financial engineering texts are very theoretical. This book is not." --Glyn Holton, Contingency Analysis

REINFORCEMENT LEARNING AND STOCHASTIC OPTIMIZATION Clearing the jungle of stochastic optimization Sequential decision problems, which consist of “decision, information, decision, information,” are ubiquitous, spanning virtually every human activity ranging from business applications, health (personal and public health, and medical decision making), energy, the sciences, all fields of engineering, finance, and e-commerce. The diversity of applications attracted the attention of at least 15 distinct fields of research, using eight distinct notational systems which produced a vast array of analytical tools. A byproduct is that powerful tools developed in one community may be unknown to other communities. Reinforcement Learning and Stochastic Optimization offers a single canonical framework that can model any sequential decision problem using five core components: state variables, decision variables, exogenous information variables, transition function, and objective function. This book highlights twelve types of uncertainty that might enter any model and pulls together the diverse set of methods for making decisions, known as policies, into four fundamental classes that span every method suggested in the academic literature or used in practice. Reinforcement Learning and Stochastic Optimization is the first book to provide a balanced treatment of the different methods for modeling and solving sequential decision problems, following the style used by most books on machine learning, optimization, and simulation. The presentation is designed for readers with a course in probability and statistics, and an interest in modeling and applications. Linear programming is occasionally used for specific problem classes. The book is designed for readers who are new to the field, as well as those with some background in optimization under uncertainty. Throughout this book, readers will find references to over 100 different applications, spanning pure learning problems, dynamic resource allocation problems, general state-dependent problems, and hybrid learning/resource allocation problems such as those that arose in the COVID pandemic. There are 370 exercises, organized into seven groups, ranging from review questions, modeling, computation, problem solving, theory, programming exercises and a "diary problem" that a reader chooses at the beginning of the book, and which is used as a basis for questions throughout the rest of the book.