Stochastic Modelling And Control

This book aims to provide a unified treatment of input/output modelling and of control for discrete-time dynamical systems subject to random disturbances. The results presented are of wide applica bility in control engineering, operations research, econometric modelling and many other areas. There are two distinct approaches to mathematical modelling of physical systems: a direct analysis of the physical mechanisms that comprise the process, or a 'black box' approach based on analysis of input/output data. The second approach is adopted here, although of course the properties ofthe models we study, which within the limits of linearity are very general, are also relevant to the behaviour of systems represented by such models, however they are arrived at. The type of system we are interested in is a discrete-time or sampled-data system where the relation between input and output is (at least approximately) linear and where additive random dis turbances are also present, so that the behaviour of the system must be investigated by statistical methods. After a preliminary chapter summarizing elements of probability and linear system theory, we introduce in Chapter 2 some general linear stochastic models, both in input/output and state-space form. Chapter 3 concerns filtering theory: estimation of the state of a dynamical system from noisy observations. As well as being an important topic in its own right, filtering theory provides the link, via the so-called innovations representation, between input/output models (as identified by data analysis) and state-space models, as required for much contemporary control theory.

This book aims to provide a unified treatment of input/output modelling and of control for discrete-time dynamical systems subject to random disturbances. The results presented are of wide applica bility in control engineering, operations research, econometric modelling and many other areas. There are two distinct approaches to mathematical modelling of physical systems: a direct analysis of the physical mechanisms that comprise the process, or a 'black box' approach based on analysis of input/output data. The second approach is adopted here, although of course the properties ofthe models we study, which within the limits of linearity are very general, are also relevant to the behaviour of systems represented by such models, however they are arrived at. The type of system we are interested in is a discrete-time or sampled-data system where the relation between input and output is (at least approximately) linear and where additive random dis turbances are also present, so that the behaviour of the system must be investigated by statistical methods. After a preliminary chapter summarizing elements of probability and linear system theory, we introduce in Chapter 2 some general linear stochastic models, both in input/output and state-space form. Chapter 3 concerns filtering theory: estimation of the state of a dynamical system from noisy observations. As well as being an important topic in its own right, filtering theory provides the link, via the so-called innovations representation, between input/output models (as identified by data analysis) and state-space models, as required for much contemporary control theory.

Over the past decades, although stochastic system control has been studied intensively within the field of control engineering, all the modelling and control strategies developed so far have concentrated on the performance of one or two output properties of the system. such as minimum variance control and mean value control. The general assumption used in the formulation of modelling and control strategies is that the distribution of the random signals involved is Gaussian. In this book, a set of new approaches for the control of the output probability density function of stochastic dynamic systems (those subjected to any bounded random inputs), has been developed. In this context, the purpose of control system design becomes the selection of a control signal that makes the shape of the system outputs p.d.f. as close as possible to a given distribution. The book contains material on the subjects of: - Control of single-input single-output and multiple-input multiple-output stochastic systems; - Stable adaptive control of stochastic distributions; - Model reference adaptive control; - Control of nonlinear dynamic stochastic systems; - Condition monitoring of bounded stochastic distributions; - Control algorithm design; - Singular stochastic systems. A new representation of dynamic stochastic systems is produced by using B-spline functions to descripe the output p.d.f. Advances in Industrial Control aims to report and encourage the transfer of technology in control engineering. The rapid development of control technology has an impact on all areas of the control discipline. The series offers an opportunity for researchers to present an extended exposition of new work in all aspects of industrial control.

This book offers a systematic introduction to the optimal stochastic control theory via the dynamic programming principle, which is a powerful tool to analyze control problems. First we consider completely observable control problems with finite horizons. Using a time discretization we construct a nonlinear semigroup related to the dynamic programming principle (DPP), whose generator provides the Hamilton–Jacobi–Bellman (HJB) equation, and we characterize the value function via the nonlinear semigroup, besides the viscosity solution theory. When we control not only the dynamics of a system but also the terminal time of its evolution, control-stopping problems arise. This problem is treated in the same frameworks, via the nonlinear semigroup. Its results are applicable to the American option price problem. Zero-sum two-player time-homogeneous stochastic differential games and viscosity solutions of the Isaacs equations arising from such games are studied via a nonlinear semigroup related to DPP (the min-max principle, to be precise). Using semi-discretization arguments, we construct the nonlinear semigroups whose generators provide lower and upper Isaacs equations. Concerning partially observable control problems, we refer to stochastic parabolic equations driven by colored Wiener noises, in particular, the Zakai equation. The existence and uniqueness of solutions and regularities as well as Itô's formula are stated. A control problem for the Zakai equations has a nonlinear semigroup whose generator provides the HJB equation on a Banach space. The value function turns out to be a unique viscosity solution for the HJB equation under mild conditions. This edition provides a more generalized treatment of the topic than does the earlier book Lectures on Stochastic Control Theory (ISI Lecture Notes 9), where time-homogeneous cases are dealt with. Here, for finite time-horizon control problems, DPP was formulated as a one-parameter nonlinear semigroup, whose generator provides the HJB equation, by using a time-discretization method. The semigroup corresponds to the value function and is characterized as the envelope of Markovian transition semigroups of responses for constant control processes. Besides finite time-horizon controls, the book discusses control-stopping problems in the same frameworks.

Applied Stochastic Models and Control for Finance and Insurance presents at an introductory level some essential stochastic models applied in economics, finance and insurance. Markov chains, random walks, stochastic differential equations and other stochastic processes are used throughout the book and systematically applied to economic and financial applications. In addition, a dynamic programming framework is used to deal with some basic optimization problems. The book begins by introducing problems of economics, finance and insurance which involve time, uncertainty and risk. A number of cases are treated in detail, spanning risk management, volatility, memory, the time structure of preferences, interest rates and yields, etc. The second and third chapters provide an introduction to stochastic models and their application. Stochastic differential equations and stochastic calculus are presented in an intuitive manner, and numerous applications and exercises are used to facilitate their understanding and their use in Chapter 3. A number of other processes which are increasingly used in finance and insurance are introduced in Chapter 4. In the fifth chapter, ARCH and GARCH models are presented and their application to modeling volatility is emphasized. An outline of decision-making procedures is presented in Chapter 6. Furthermore, we also introduce the essentials of stochastic dynamic programming and control, and provide first steps for the student who seeks to apply these techniques. Finally, in Chapter 7, numerical techniques and approximations to stochastic processes are examined. This book can be used in business, economics, financial engineering and decision sciences schools for second year Master's students, as well as in a number of courses widely given in departments of statistics, systems and decision sciences.

This book is devoted to a specific problem in the general theory of automatic control -- sequential control under conditions of incomplete information. The main results concern the case in which at each moment of (continuous) time only a finite number of controls are admissible and the results of control action are represented by realizations of random variables whose distributions at a given control correspond to one of several alternative hypotheses. The analysis is conducted in a Bayesian framework.

As is well known, Pontryagin's maximum principle and Bellman's dynamic programming are the two principal and most commonly used approaches in solving stochastic optimal control problems. * An interesting phenomenon one can observe from the literature is that these two approaches have been developed separately and independently. Since both methods are used to investigate the same problems, a natural question one will ask is the fol lowing: (Q) What is the relationship betwccn the maximum principlc and dy namic programming in stochastic optimal controls? There did exist some researches (prior to the 1980s) on the relationship between these two. Nevertheless, the results usually werestated in heuristic terms and proved under rather restrictive assumptions, which were not satisfied in most cases. In the statement of a Pontryagin-type maximum principle there is an adjoint equation, which is an ordinary differential equation (ODE) in the (finite-dimensional) deterministic case and a stochastic differential equation (SDE) in the stochastic case. The system consisting of the adjoint equa tion, the original state equation, and the maximum condition is referred to as an (extended) Hamiltonian system. On the other hand, in Bellman's dynamic programming, there is a partial differential equation (PDE), of first order in the (finite-dimensional) deterministic case and of second or der in the stochastic case. This is known as a Hamilton-Jacobi-Bellman (HJB) equation.

This volume collects papers, based on invited talks given at the IMA workshop in Modeling, Stochastic Control, Optimization, and Related Applications, held at the Institute for Mathematics and Its Applications, University of Minnesota, during May and June, 2018. There were four week-long workshops during the conference. They are (1) stochastic control, computation methods, and applications, (2) queueing theory and networked systems, (3) ecological and biological applications, and (4) finance and economics applications. For broader impacts, researchers from different fields covering both theoretically oriented and application intensive areas were invited to participate in the conference. It brought together researchers from multi-disciplinary communities in applied mathematics, applied probability, engineering, biology, ecology, and networked science, to review, and substantially update most recent progress. As an archive, this volume presents some of the highlights of the workshops, and collect papers covering a broad range of topics.