Analysis And Optimisation Of Stochastic Systems

Optimal stochastic control; Stochastic optimisation; Stochastic processes; Algorithms; Information; Parameter estimation; Applications.

Optimization of Stochastic Systems is an outgrowth of class notes of a graduate level seminar on optimization of stochastic systems. Most of the material in the book was taught for the first time during the 1965 Spring Semester while the author was visiting the Department of Electrical Engineering, University of California, Berkeley. The revised and expanded material was presented at the Department of Engineering, University of California, Los Angeles during the 1965 Fall Semester. The systems discussed in the book are mostly assumed to be of discrete-time type with continuous state variables taking values in some subsets of Euclidean spaces. There is another class of systems in which state variables are assumed to take on at most a denumerable number of values, i.e., these systems are of discrete-time discrete-space type. Although the problems associated with the latter class of systems are many and interesting, andalthough they are amenable to deep analysis on such topics as the limiting behaviors of state variables as time indexes increase to infinity, this class of systems is not included here, partly because there are many excellent books on the subjects and partly because inclusion of these materials would easily double the size of the book.

Stochastic programming is the study of procedures for decision making under the presence of uncertainties and risks. Stochastic programming approaches have been successfully used in a number of areas such as energy and production planning, telecommunications, and transportation. Recently, the practical experience gained in stochastic programming has been expanded to a much larger spectrum of applications including financial modeling, risk management, and probabilistic risk analysis. Major topics in this volume include: (1) advances in theory and implementation of stochastic programming algorithms; (2) sensitivity analysis of stochastic systems; (3) stochastic programming applications and other related topics. Audience: Researchers and academies working in optimization, computer modeling, operations research and financial engineering. The book is appropriate as supplementary reading in courses on optimization and financial engineering.

From the Preface The first edition of this book was written mainly for audiences with physical science and engineering backgrounds. Nevertheless, it reached some readers with economic and management science training. Analytical training of graduate students in economics and management sciences had progressed much in the last 20 years, and many new research results and optimization algorithms have also become available. My own interest in the meantime has shifted to the analysis of dynamics and optimization problems of economic and management science origin. With these developments and changes, I decided to rewrite much of the first edition to make it more accessible to graduate students and professionals in social sciences. I have also incorporated some new analytic tools that I deem useful in analyzing the dynamic and stochastic problems which confront these readers. I hope that my efforts successfully bring intertemporal optimization problems closer to economics professionals. New topics introduced into this second edition appear mostly in Chapters 2, 4, 5, 6, and 8. Martingales and martingale differences are introduced early in Chapter 2. Some limit theorems and asymptotic properties of linear state space models driven by martingale differences are presented. Because many excellent books are available on martingales and their limit theorems, derivations and proofs are mostly sketchy, and readers are referred to these sources. The results in Chapteer 2 are applied in Chapters 5, 6, and 8, among other places. The notion of dynamic aggregation and its relation to cointegration and error-correction models are developed in Chapter 4. Some recursive parameter estimation schemes and their statistical properties are included in Chapters 5 and 6. Here again, books devoted entirely to these topics are available in the literature, and much had to be omitted to keep the second edition to a manageable size. In an appendix to Chapter 7, a potentially very powerful tool in proving convergence of adaptive schemes is outlined. Rational expectations models and their solution methods are developed in Chapter 8 because of their wide-spread interest to economists. A very important class of problems in sequential decision problems revolves around questions of approximating nonlinear dynamics or more generally complex situations with a sequence of less complex ones. Chapter 9 does not begin to do justice to this class of problems but is included as being suggestive of works to be done. When I first started contemplating the revision of the first edition, I benefited from a list of excellent suggestions from Rick van der Ploeg, though I did not necessarily incorporate all of his suggestions. Conversations with Thomas Sargent and Victor Solo were useful in organizing the material into the form of the second edition. I also benefited from discussions with Hashem Pesaran and correspondences with L. Broze in finalizing Chapter 8. Some material in this book was used as lecture notes in a graduate course in the Department of Economics, University of California, Los Angeles, the winter quarter of 1987. I thank the participants in the course for many useful comments. Key Features * This major revision of the First Edition addresses optimization problems stated in stochastic difference equations, which often contain uncertain or randomly varying parameters * Presents a set of concepts and techniques useful in analyzing or controlling stochastic dynamic processes, with possible incompletely specified characteristics * It discusses basic system properties such as: * Stability and observability * Dynamic programming formulations of optimal and adaptive control problems * Parameter estimation schemes and their convergence behavior * Solution methods for rational expectations models using martingale differences

Optimization of Stochastic Systems

A unified and rigorous treatment of the associated stochastic optimization problems is provided and recent advances in perturbation theory encompassed. Throughout the book emphasis is upon concepts rather than mathematical completeness with the advantage that the reader only requires a basic knowledge of probability, statistics and optimization.

In view of Professor Wendell Fleming's many fundamental contributions, his profound influence on the mathematical and systems theory communi ties, his service to the profession, and his dedication to mathematics, we have invited a number of leading experts in the fields of control, optimiza tion, and stochastic systems to contribute to this volume in his honor on the occasion of his 70th birthday. These papers focus on various aspects of stochastic analysis, control theory and optimization, and applications. They include authoritative expositions and surveys as well as research papers on recent and important issues. The papers are grouped according to the following four major themes: (1) large deviations, risk sensitive and Hoc control, (2) partial differential equations and viscosity solutions, (3) stochastic control, filtering and parameter esti mation, and (4) mathematical finance and other applications. We express our deep gratitude to all of the authors for their invaluable contributions, and to the referees for their careful and timely reviews. We thank Harold Kushner for having graciously agreed to undertake the task of writing the foreword. Particular thanks go to H. Thomas Banks for his help, advice and suggestions during the entire preparation process, as well as for the generous support of the Center for Research in Scientific Computation. The assistance from the Birkhauser professional staff is also greatly appreciated.

This book introduces some advanced topics in probability theories ? both pure and applied ? is divided into two parts. The first part deals with the analysis of stochastic dynamical systems, in terms of Gaussian processes, white noise theory, and diffusion processes. The second part of the book discusses some up-to-date applications of optimization theories, martingale measure theories, reliability theories, stochastic filtering theories and stochastic algorithms towards mathematical finance issues such as option pricing and hedging, bond market analysis, volatility studies and asset trading modeling.