Optimization Algorithms On Matrix Manifolds

Many problems in the sciences and engineering can be rephrased as optimization problems on matrix search spaces endowed with a so-called manifold structure. This book shows how to exploit the special structure of such problems to develop efficient numerical algorithms. It places careful emphasis on both the numerical formulation of the algorithm and its differential geometric abstraction--illustrating how good algorithms draw equally from the insights of differential geometry, optimization, and numerical analysis. Two more theoretical chapters provide readers with the background in differential geometry necessary to algorithmic development. In the other chapters, several well-known optimization methods such as steepest descent and conjugate gradients are generalized to abstract manifolds. The book provides a generic development of each of these methods, building upon the material of the geometric chapters. It then guides readers through the calculations that turn these geometrically formulated methods into concrete numerical algorithms. The state-of-the-art algorithms given as examples are competitive with the best existing algorithms for a selection of eigenspace problems in numerical linear algebra. Optimization Algorithms on Matrix Manifolds offers techniques with broad applications in linear algebra, signal processing, data mining, computer vision, and statistical analysis. It can serve as a graduate-level textbook and will be of interest to applied mathematicians, engineers, and computer scientists.

Many problems in the sciences and engineering can be rephrased as optimization problems on matrix search spaces endowed with a so-called manifold structure. This book shows how to exploit the special structure of such problems to develop efficient numerical algorithms. It places careful emphasis on both the numerical formulation of the algorithm and its differential geometric abstraction--illustrating how good algorithms draw equally from the insights of differential geometry, optimization, and numerical analysis. Two more theoretical chapters provide readers with the background in differential geometry necessary to algorithmic development. In the other chapters, several well-known optimization methods such as steepest descent and conjugate gradients are generalized to abstract manifolds. The book provides a generic development of each of these methods, building upon the material of the geometric chapters. It then guides readers through the calculations that turn these geometrically formulated methods into concrete numerical algorithms. The state-of-the-art algorithms given as examples are competitive with the best existing algorithms for a selection of eigenspace problems in numerical linear algebra. Optimization Algorithms on Matrix Manifolds offers techniques with broad applications in linear algebra, signal processing, data mining, computer vision, and statistical analysis. It can serve as a graduate-level textbook and will be of interest to applied mathematicians, engineers, and computer scientists.

This brief describes the basics of Riemannian optimization—optimization on Riemannian manifolds—introduces algorithms for Riemannian optimization problems, discusses the theoretical properties of these algorithms, and suggests possible applications of Riemannian optimization to problems in other fields. To provide the reader with a smooth introduction to Riemannian optimization, brief reviews of mathematical optimization in Euclidean spaces and Riemannian geometry are included. Riemannian optimization is then introduced by merging these concepts. In particular, the Euclidean and Riemannian conjugate gradient methods are discussed in detail. A brief review of recent developments in Riemannian optimization is also provided. Riemannian optimization methods are applicable to many problems in various fields. This brief discusses some important applications including the eigenvalue and singular value decompositions in numerical linear algebra, optimal model reduction in control engineering, and canonical correlation analysis in statistics.

This book covers different, current research directions in the context of variational methods for non-linear geometric data. Each chapter is authored by leading experts in the respective discipline and provides an introduction, an overview and a description of the current state of the art. Non-linear geometric data arises in various applications in science and engineering. Examples of nonlinear data spaces are diverse and include, for instance, nonlinear spaces of matrices, spaces of curves, shapes as well as manifolds of probability measures. Applications can be found in biology, medicine, product engineering, geography and computer vision for instance. Variational methods on the other hand have evolved to being amongst the most powerful tools for applied mathematics. They involve techniques from various branches of mathematics such as statistics, modeling, optimization, numerical mathematics and analysis. The vast majority of research on variational methods, however, is focused on data in linear spaces. Variational methods for non-linear data is currently an emerging research topic. As a result, and since such methods involve various branches of mathematics, there is a plethora of different, recent approaches dealing with different aspects of variational methods for nonlinear geometric data. Research results are rather scattered and appear in journals of different mathematical communities. The main purpose of the book is to account for that by providing, for the first time, a comprehensive collection of different research directions and existing approaches in this context. It is organized in a way that leading researchers from the different fields provide an introductory overview of recent research directions in their respective discipline. As such, the book is a unique reference work for both newcomers in the field of variational methods for non-linear geometric data, as well as for established experts that aim at to exploit new research directions or collaborations. Chapter 9 of this book is available open access under a CC BY 4.0 license at link.springer.com.

Mathematical optimization encompasses both a rich and rapidly evolving body of fundamental theory, and a variety of exciting applications in science and engineering. The present book contains a careful selection of articles on recent advances in optimization theory, numerical methods, and their applications in engineering. It features in particular new methods and applications in the fields of optimal control, PDE-constrained optimization, nonlinear optimization, and convex optimization. The authors of this volume took part in the 14th Belgian-French-German Conference on Optimization (BFG09) organized in Leuven, Belgium, on September 14-18, 2009. The volume contains a selection of reviewed articles contributed by the conference speakers as well as three survey articles by plenary speakers and two papers authored by the winners of the best talk and best poster prizes awarded at BFG09. Researchers and graduate students in applied mathematics, computer science, and many branches of engineering will find in this book an interesting and useful collection of recent ideas on the methods and applications of optimization.

Author has written several excellent Springer books.; This book is a sequel to Introduction to Topological Manifolds; Careful and illuminating explanations, excellent diagrams and exemplary motivation; Includes short preliminary sections before each section explaining what is ahead and why

This text focuses on developing an intimate acquaintance with the geometric meaning of curvature and thereby introduces and demonstrates all the main technical tools needed for a more advanced course on Riemannian manifolds. It covers proving the four most fundamental theorems relating curvature and topology: the Gauss-Bonnet Theorem, the Cartan-Hadamard Theorem, Bonnet’s Theorem, and a special case of the Cartan-Ambrose-Hicks Theorem.

This book presents a selection of the most recent algorithmic advances in Riemannian geometry in the context of machine learning, statistics, optimization, computer vision, and related fields. The unifying theme of the different chapters in the book is the exploitation of the geometry of data using the mathematical machinery of Riemannian geometry. As demonstrated by all the chapters in the book, when the data is intrinsically non-Euclidean, the utilization of this geometrical information can lead to better algorithms that can capture more accurately the structures inherent in the data, leading ultimately to better empirical performance. This book is not intended to be an encyclopedic compilation of the applications of Riemannian geometry. Instead, it focuses on several important research directions that are currently actively pursued by researchers in the field. These include statistical modeling and analysis on manifolds,optimization on manifolds, Riemannian manifolds and kernel methods, and dictionary learning and sparse coding on manifolds. Examples of applications include novel algorithms for Monte Carlo sampling and Gaussian Mixture Model fitting, 3D brain image analysis,image classification, action recognition, and motion tracking.