Newton Methods

This book deals with the efficient numerical solution of challenging nonlinear problems in science and engineering, both in finite and in infinite dimension. Its focus is on local and global Newton methods for direct problems or Gauss-Newton methods for inverse problems. Lots of numerical illustrations, comparison tables, and exercises make the text useful in computational mathematics classes. At the same time, the book opens many directions for possible future research.

This book deals with the efficient numerical solution of challenging nonlinear problems in science and engineering, both in finite dimension (algebraic systems) and in infinite dimension (ordinary and partial differential equations). Its focus is on local and global Newton methods for direct problems or Gauss-Newton methods for inverse problems. The term 'affine invariance' means that the presented algorithms and their convergence analysis are invariant under one out of four subclasses of affine transformations of the problem to be solved. Compared to traditional textbooks, the distinguishing affine invariance approach leads to shorter theorems and proofs and permits the construction of fully adaptive algorithms. Lots of numerical illustrations, comparison tables, and exercises make the text useful in computational mathematics classes. At the same time, the book opens many directions for possible future research.

This book presents comprehensive state-of-the-art theoretical analysis of the fundamental Newtonian and Newtonian-related approaches to solving optimization and variational problems. A central focus is the relationship between the basic Newton scheme for a given problem and algorithms that also enjoy fast local convergence. The authors develop general perturbed Newtonian frameworks that preserve fast convergence and consider specific algorithms as particular cases within those frameworks, i.e., as perturbations of the associated basic Newton iterations. This approach yields a set of tools for the unified treatment of various algorithms, including some not of the Newton type per se. Among the new subjects addressed is the class of degenerate problems. In particular, the phenomenon of attraction of Newton iterates to critical Lagrange multipliers and its consequences as well as stabilized Newton methods for variational problems and stabilized sequential quadratic programming for optimization. This volume will be useful to researchers and graduate students in the fields of optimization and variational analysis.

This self-contained treatment offers a contemporary and systematic development of the theory and application of Newton methods, which are undoubtedly the most effective tools for solving equations appearing in computational sciences. Its focal point resides in an exhaustive analysis of the convergence properties of several Newton variants used in connection to specific real life problems originated from astrophysics, engineering, mathematical economics and other applied areas. What distinguishes this book from others is the fact that the weak convergence conditions inaugurated here allow for a wider applicability of Newton methods; finer error bounds on the distances involved, and a more precise information on the location of the solution. These factors make this book ideal for researchers, practitioners and students.

Semismooth Newton methods are a modern class of remarkably powerful and versatile algorithms for solving constrained optimization problems with partial differential equations (PDEs), variational inequalities, and related problems. This book provides a comprehensive presentation of these methods in function spaces, striking a balance between thoroughly developed theory and numerical applications. Although largely self-contained, the book also covers recent developments in the field, such as state-constrained problems, and offers new material on topics such as improved mesh independence results. The theory and methods are applied to a range of practically important problems, including: optimal control of nonlinear elliptic differential equations, obstacle problems, and flow control of instationary Navier-Stokes fluids. In addition, the author covers adjoint-based derivative computation and the efficient solution of Newton systems by multigrid and preconditioned iterative methods.

This book on Newton's method is a user-oriented guide to algorithms and implementation. In just over 100 pages, it shows, via algorithms in pseudocode, in MATLAB, and with several examples, how one can choose an appropriate Newton-type method for a given problem, diagnose problems, and write an efficient solver or apply one written by others. It contains trouble-shooting guides to the major algorithms, their most common failure modes, and the likely causes of failure. It also includes many worked-out examples (available on the SIAM website) in pseudocode and a collection of MATLAB codes, allowing readers to experiment with the algorithms easily and implement them in other languages.