Infinite Horizon Optimal Control

This monograph deals with various classes of deterministic continuous time optimal control problems wh ich are defined over unbounded time intervala. For these problems, the performance criterion is described by an improper integral and it is possible that, when evaluated at a given admissible element, this criterion is unbounded. To cope with this divergence new optimality concepts; referred to here as "overtaking", "weakly overtaking", "agreeable plans", etc. ; have been proposed. The motivation for studying these problems arisee primarily from the economic and biological aciences where models of this nature arise quite naturally since no natural bound can be placed on the time horizon when one considers the evolution of the state of a given economy or species. The reeponsibility for the introduction of this interesting class of problems rests with the economiste who first studied them in the modeling of capital accumulation processes. Perhaps the earliest of these was F. Ramsey who, in his seminal work on a theory of saving in 1928, considered a dynamic optimization model defined on an infinite time horizon. Briefly, this problem can be described as a "Lagrange problem with unbounded time interval". The advent of modern control theory, particularly the formulation of the famoue Maximum Principle of Pontryagin, has had a considerable impact on the treatment of these models as well as optimization theory in general.

​​​​In this book the authors take a rigorous look at the infinite-horizon discrete-time optimal control theory from the viewpoint of Pontryagin’s principles. Several Pontryagin principles are described which govern systems and various criteria which define the notions of optimality, along with a detailed analysis of how each Pontryagin principle relate to each other. The Pontryagin principle is examined in a stochastic setting and results are given which generalize Pontryagin’s principles to multi-criteria problems. ​Infinite-Horizon Optimal Control in the Discrete-Time Framework is aimed toward researchers and PhD students in various scientific fields such as mathematics, applied mathematics, economics, management, sustainable development (such as, of fisheries and of forests), and Bio-medical sciences who are drawn to infinite-horizon discrete-time optimal control problems.

This book gathers the most essential results, including recent ones, on linear-quadratic optimal control problems, which represent an important aspect of stochastic control. It presents the results in the context of finite and infinite horizon problems, and discusses a number of new and interesting issues. Further, it precisely identifies, for the first time, the interconnections between three well-known, relevant issues – the existence of optimal controls, solvability of the optimality system, and solvability of the associated Riccati equation. Although the content is largely self-contained, readers should have a basic grasp of linear algebra, functional analysis and stochastic ordinary differential equations. The book is mainly intended for senior undergraduate and graduate students majoring in applied mathematics who are interested in stochastic control theory. However, it will also appeal to researchers in other related areas, such as engineering, management, finance/economics and the social sciences.

This book is devoted to the study of the turnpike phenomenon and describes the existence of solutions for a large variety of infinite horizon optimal control classes of problems. Chapter 1 provides introductory material on turnpike properties. Chapter 2 studies the turnpike phenomenon for discrete-time optimal control problems. The turnpike properties of autonomous problems with extended-value integrands are studied in Chapter 3. Chapter 4 focuses on large classes of infinite horizon optimal control problems without convexity (concavity) assumptions. In Chapter 5, the turnpike results for a class of dynamic discrete-time two-player zero-sum game are proven. This thorough exposition will be very useful for mathematicians working in the fields of optimal control, the calculus of variations, applied functional analysis and infinite horizon optimization. It may also be used as a primary text in a graduate course in optimal control or as supplementary text for a variety of courses in other disciplines. Researchers in other fields such as economics and game theory, where turnpike properties are well known, will also find this Work valuable.

This book offers readers a thorough and rigorous introduction to nonlinear model predictive control (NMPC) for discrete-time and sampled-data systems. NMPC schemes with and without stabilizing terminal constraints are detailed, and intuitive examples illustrate the performance of different NMPC variants. NMPC is interpreted as an approximation of infinite-horizon optimal control so that important properties like closed-loop stability, inverse optimality and suboptimality can be derived in a uniform manner. These results are complemented by discussions of feasibility and robustness. An introduction to nonlinear optimal control algorithms yields essential insights into how the nonlinear optimization routine—the core of any nonlinear model predictive controller—works. Accompanying software in MATLAB® and C++ (downloadable from extras.springer.com/), together with an explanatory appendix in the book itself, enables readers to perform computer experiments exploring the possibilities and limitations of NMPC. The second edition has been substantially rewritten, edited and updated to reflect the significant advances that have been made since the publication of its predecessor, including: • a new chapter on economic NMPC relaxing the assumption that the running cost penalizes the distance to a pre-defined equilibrium; • a new chapter on distributed NMPC discussing methods which facilitate the control of large-scale systems by splitting up the optimization into smaller subproblems; • an extended discussion of stability and performance using approximate updates rather than full optimization; • replacement of the pivotal sufficient condition for stability without stabilizing terminal conditions with a weaker alternative and inclusion of an alternative and much simpler proof in the analysis; and • further variations and extensions in response to suggestions from readers of the first edition. Though primarily aimed at academic researchers and practitioners working in control and optimization, the text is self-contained, featuring background material on infinite-horizon optimal control and Lyapunov stability theory that also makes it accessible for graduate students in control engineering and applied mathematics.

Neural Approximations for Optimal Control and Decision provides a comprehensive methodology for the approximate solution of functional optimization problems using neural networks and other nonlinear approximators where the use of traditional optimal control tools is prohibited by complicating factors like non-Gaussian noise, strong nonlinearities, large dimension of state and control vectors, etc. Features of the text include: • a general functional optimization framework; • thorough illustration of recent theoretical insights into the approximate solutions of complex functional optimization problems; • comparison of classical and neural-network based methods of approximate solution; • bounds to the errors of approximate solutions; • solution algorithms for optimal control and decision in deterministic or stochastic environments with perfect or imperfect state measurements over a finite or infinite time horizon and with one decision maker or several; • applications of current interest: routing in communications networks, traffic control, water resource management, etc.; and • numerous, numerically detailed examples. The authors’ diverse backgrounds in systems and control theory, approximation theory, machine learning, and operations research lend the book a range of expertise and subject matter appealing to academics and graduate students in any of those disciplines together with computer science and other areas of engineering.