Introduction To Nonsmooth Optimization

This book is the first easy-to-read text on nonsmooth optimization (NSO, not necessarily differentiable optimization). Solving these kinds of problems plays a critical role in many industrial applications and real-world modeling systems, for example in the context of image denoising, optimal control, neural network training, data mining, economics and computational chemistry and physics. The book covers both the theory and the numerical methods used in NSO and provide an overview of different problems arising in the field. It is organized into three parts: 1. convex and nonconvex analysis and the theory of NSO; 2. test problems and practical applications; 3. a guide to NSO software. The book is ideal for anyone teaching or attending NSO courses. As an accessible introduction to the field, it is also well suited as an independent learning guide for practitioners already familiar with the basics of optimization.

This book is a self-contained elementary study for nonsmooth analysis and optimization, and their use in solution of nonsmooth optimal control problems. The first part of the book is concerned with nonsmooth differential calculus containing necessary tools for nonsmooth optimization. The second part is devoted to the methods of nonsmooth optimization and their development. A proximal bundle method for nonsmooth nonconvex optimization subject to nonsmooth constraints is constructed. In the last part nonsmooth optimization is applied to problems arising from optimal control of systems covered by partial differential equations. Several practical problems, like process control and optimal shape design problems are considered.

The goal of this book is to present the main ideas and techniques in the field of continuous smooth and nonsmooth optimization. Starting with the case of differentiable data and the classical results on constrained optimization problems, and continuing with the topic of nonsmooth objects involved in optimization theory, the book concentrates on both theoretical and practical aspects of this field. This book prepares those who are engaged in research by giving repeated insights into ideas that are subsequently dealt with and illustrated in detail.

Functional analysis has become one of the essential foundations of modern applied mathematics in the last decades, from the theory and numerical solution of differential equations, from optimization and probability theory to medical imaging and mathematical image processing. This textbook offers a compact introduction to the theory and is designed to be used during one semester, fitting exactly 26 lectures of 90 minutes each. It ranges from the topological fundamentals recalled from basic lectures on real analysis to spectral theory in Hilbert spaces. Special attention is given to the central results on dual spaces and weak convergence.

"This book concerns matter that is intrinsically difficult: convex optimization, complementarity and duality, nonsmooth analysis, linear and nonlinear programming, etc. The author has skillfully introduced these and many more concepts, and woven them into a seamless whole by retaining an easy and consistent style throughout. The book is not all the

Presents the elements of a unified approach to optimization based on 'nonsmooth analysis', a term introduced in the 1970's by the author, who is a pioneer in the field. Based on a series of lectures given at a conference at Emory University in 1986, this volume presents its subjects in a self-contained and accessible manner. The topics treated here have been in an active state of development. Focuses mainly on deterministic optimal control, the calculus of variations, and mathematical programming. In addition, it features a tutorial in nonsmooth analysis and geometry and demonstrates that the method of value function analysis via proximal normals is a powerful tool in the study of necessary conditions, sufficient conditions, controllability, and sensitivity analysis. The distinction between inductive and deductive methods, the use of Hamiltonians, the verification technique, and penalization are also emphasized.

This book treats various concepts of generalized derivatives and subdifferentials in normed spaces, their geometric counterparts and their application to optimization problems. It starts with the subdifferential of convex analysis, passes to corresponding concepts for locally Lipschitz continuous functions and then presents subdifferentials for general lower semicontinuous functions. All basic tools are presented where they are needed: this concerns separation theorems, variational and extremal principles as well as relevant parts of multifunction theory. Each chapter ends with bibliographic notes and exercises.

In the early fifties, applied mathematicians, engineers and economists started to pay c10se attention to the optimization problems in which another (lower-Ievel) optimization problem arises as a side constraint. One of the motivating factors was the concept of the Stackelberg solution in game theory, together with its economic applications. Other problems have been encountered in the seventies in natural sciences and engineering. Many of them are of practical importance and have been extensively studied, mainly from the theoretical point of view. Later, applications to mechanics and network design have lead to an extension of the problem formulation: Constraints in form of variation al inequalities and complementarity problems were also admitted. The term "generalized bi level programming problems" was used at first but later, probably in Harker and Pang, 1988, a different terminology was introduced: Mathematical programs with equilibrium constraints, or simply, MPECs. In this book we adhere to MPEC terminology. A large number of papers deals with MPECs but, to our knowledge, there is only one monograph (Luo et al. , 1997). This monograph concentrates on optimality conditions and numerical methods. Our book is oriented similarly, but we focus on those MPECs which can be treated by the implicit programming approach: the equilibrium constraint locally defines a certain implicit function and allows to convert the problem into a mathematical program with a nonsmooth objective.