Optimization Of Stochastic Systems

Optimization of Stochastic Systems is an outgrowth of class notes of a graduate level seminar on optimization of stochastic systems. Most of the material in the book was taught for the first time during the 1965 Spring Semester while the author was visiting the Department of Electrical Engineering, University of California, Berkeley. The revised and expanded material was presented at the Department of Engineering, University of California, Los Angeles during the 1965 Fall Semester. The systems discussed in the book are mostly assumed to be of discrete-time type with continuous state variables taking values in some subsets of Euclidean spaces. There is another class of systems in which state variables are assumed to take on at most a denumerable number of values, i.e., these systems are of discrete-time discrete-space type. Although the problems associated with the latter class of systems are many and interesting, andalthough they are amenable to deep analysis on such topics as the limiting behaviors of state variables as time indexes increase to infinity, this class of systems is not included here, partly because there are many excellent books on the subjects and partly because inclusion of these materials would easily double the size of the book.

Optimization of Stochastic Systems

Optimal stochastic control; Stochastic optimisation; Stochastic processes; Algorithms; Information; Parameter estimation; Applications.

From the Preface The first edition of this book was written mainly for audiences with physical science and engineering backgrounds. Nevertheless, it reached some readers with economic and management science training. Analytical training of graduate students in economics and management sciences had progressed much in the last 20 years, and many new research results and optimization algorithms have also become available. My own interest in the meantime has shifted to the analysis of dynamics and optimization problems of economic and management science origin. With these developments and changes, I decided to rewrite much of the first edition to make it more accessible to graduate students and professionals in social sciences. I have also incorporated some new analytic tools that I deem useful in analyzing the dynamic and stochastic problems which confront these readers. I hope that my efforts successfully bring intertemporal optimization problems closer to economics professionals. New topics introduced into this second edition appear mostly in Chapters 2, 4, 5, 6, and 8. Martingales and martingale differences are introduced early in Chapter 2. Some limit theorems and asymptotic properties of linear state space models driven by martingale differences are presented. Because many excellent books are available on martingales and their limit theorems, derivations and proofs are mostly sketchy, and readers are referred to these sources. The results in Chapteer 2 are applied in Chapters 5, 6, and 8, among other places. The notion of dynamic aggregation and its relation to cointegration and error-correction models are developed in Chapter 4. Some recursive parameter estimation schemes and their statistical properties are included in Chapters 5 and 6. Here again, books devoted entirely to these topics are available in the literature, and much had to be omitted to keep the second edition to a manageable size. In an appendix to Chapter 7, a potentially very powerful tool in proving convergence of adaptive schemes is outlined. Rational expectations models and their solution methods are developed in Chapter 8 because of their wide-spread interest to economists. A very important class of problems in sequential decision problems revolves around questions of approximating nonlinear dynamics or more generally complex situations with a sequence of less complex ones. Chapter 9 does not begin to do justice to this class of problems but is included as being suggestive of works to be done. When I first started contemplating the revision of the first edition, I benefited from a list of excellent suggestions from Rick van der Ploeg, though I did not necessarily incorporate all of his suggestions. Conversations with Thomas Sargent and Victor Solo were useful in organizing the material into the form of the second edition. I also benefited from discussions with Hashem Pesaran and correspondences with L. Broze in finalizing Chapter 8. Some material in this book was used as lecture notes in a graduate course in the Department of Economics, University of California, Los Angeles, the winter quarter of 1987. I thank the participants in the course for many useful comments. Key Features * This major revision of the First Edition addresses optimization problems stated in stochastic difference equations, which often contain uncertain or randomly varying parameters * Presents a set of concepts and techniques useful in analyzing or controlling stochastic dynamic processes, with possible incompletely specified characteristics * It discusses basic system properties such as: * Stability and observability * Dynamic programming formulations of optimal and adaptive control problems * Parameter estimation schemes and their convergence behavior * Solution methods for rational expectations models using martingale differences

The DMV seminar "Stochastische Approximation und Optimierung zufalliger Systeme" was held at Blaubeuren, 28. 5. -4. 6. 1989. The goal was to give an approach to theory and application of stochas tic approximation in view of optimization problems, especially in engineering systems. These notes are based on the seminar lectures. They consist of three parts: I. Foundations of stochastic approximation (H. Walk); n. Applicational aspects of stochastic approximation (G. PHug); In. Applications to adaptation :ugorithms (L. Ljung). The prerequisites for reading this book are basic knowledge in probability, mathematical statistics, optimization. We would like to thank Prof. M. Barner and Prof. G. Fischer for the or ganization of the seminar. We also thank the participants for their cooperation and our assistants and secretaries for typing the manuscript. November 1991 L. Ljung, G. PHug, H. Walk Table of contents I Foundations of stochastic approximation (H. Walk) §1 Almost sure convergence of stochastic approximation procedures 2 §2 Recursive methods for linear problems 17 §3 Stochastic optimization under stochastic constraints 22 §4 A learning model; recursive density estimation 27 §5 Invariance principles in stochastic approximation 30 §6 On the theory of large deviations 43 References for Part I 45 11 Applicational aspects of stochastic approximation (G. PHug) §7 Markovian stochastic optimization and stochastic approximation procedures 53 §8 Asymptotic distributions 71 §9 Stopping times 79 §1O Applications of stochastic approximation methods 80 References for Part II 90 III Applications to adaptation algorithms (L.

This volume provides a general overview of discrete- and continuous-time Markov control processes and stochastic games, along with a look at the range of applications of stochastic control and some of its recent theoretical developments. These topics include various aspects of dynamic programming, approximation algorithms, and infinite-dimensional linear programming. In all, the work comprises 18 carefully selected papers written by experts in their respective fields. Optimization, Control, and Applications of Stochastic Systems will be a valuable resource for all practitioners, researchers, and professionals in applied mathematics and operations research who work in the areas of stochastic control, mathematical finance, queueing theory, and inventory systems. It may also serve as a supplemental text for graduate courses in optimal control and dynamic games.

This books covers the broad range of research in stochastic models and optimization. Applications presented include networks, financial engineering, production planning, and supply chain management. Each contribution is aimed at graduate students working in operations research, probability, and statistics.