Matrix Algorithms

This volume is the first in a self-contained five-volume series devoted to matrix algorithms. It focuses on the computation of matrix decompositions--that is, the factorization of matrices into products of similar ones. The first two chapters provide the required background from mathematics and computer science needed to work effectively in matrix computations. The remaining chapters are devoted to the LU and QR decompositions--their computation and applications. The singular value decomposition is also treated, although algorithms for its computation will appear in the second volume of the series. The present volume contains 65 algorithms formally presented in pseudocode. Other volumes in the series will treat eigensystems, iterative methods, sparse matrices, and structured problems. The series is aimed at the nonspecialist who needs more than black-box proficiency with matrix computations. To give the series focus, the emphasis is on algorithms, their derivation, and their analysis. The reader is assumed to have a knowledge of elementary analysis and linear algebra and a reasonable amount of programming experience, typically that of the beginning graduate engineer or the undergraduate in an honors program. Strictly speaking, the individual volumes are not textbooks, although they are intended to teach, the guiding principle being that if something is worth explaining, it is worth explaining fully. This has necessarily restricted the scope of the series, but the selection of topics should give the reader a sound basis for further study.

This self-contained monograph presents matrix algorithms and their analysis. The new technique enables not only the solution of linear systems but also the approximation of matrix functions, e.g., the matrix exponential. Other applications include the solution of matrix equations, e.g., the Lyapunov or Riccati equation. The required mathematical background can be found in the appendix. The numerical treatment of fully populated large-scale matrices is usually rather costly. However, the technique of hierarchical matrices makes it possible to store matrices and to perform matrix operations approximately with almost linear cost and a controllable degree of approximation error. For important classes of matrices, the computational cost increases only logarithmically with the approximation error. The operations provided include the matrix inversion and LU decomposition. Since large-scale linear algebra problems are standard in scientific computing, the subject of hierarchical matrices is of interest to scientists in computational mathematics, physics, chemistry and engineering.

This volume is the first in a self-contained five-volume series devoted to matrix algorithms. It focuses on the computation of matrix decompositions--that is, the factorization of matrices into products of similar ones. The first two chapters provide the required background from mathematics and computer science needed to work effectively in matrix computations. The remaining chapters are devoted to the LU and QR decompositions--their computation and applications. The singular value decomposition is also treated, although algorithms for its computation will appear in the second volume of the series. The present volume contains 65 algorithms formally presented in pseudocode. Other volumes in the series will treat eigensystems, iterative methods, sparse matrices, and structured problems. The series is aimed at the nonspecialist who needs more than black-box proficiency with matrix computations. To give the series focus, the emphasis is on algorithms, their derivation, and their analysis. The reader is assumed to have a knowledge of elementary analysis and linear algebra and a reasonable amount of programming experience, typically that of the beginning graduate engineer or the undergraduate in an honors program. Strictly speaking, the individual volumes are not textbooks, although they are intended to teach, the guiding principle being that if something is worth explaining, it is worth explaining fully. This has necessarily restricted the scope of the series, but the selection of topics should give the reader a sound basis for further study.

Matrix Algorithms in MATLAB focuses on the MATLAB code implementations of matrix algorithms. The MATLAB codes presented in the book are tested with thousands of runs of MATLAB randomly generated matrices, and the notation in the book follows the MATLAB style to ensure a smooth transition from formulation to the code, with MATLAB codes discussed in this book kept to within 100 lines for the sake of clarity. The book provides an overview and classification of the interrelations of various algorithms, as well as numerous examples to demonstrate code usage and the properties of the presented algorithms. Despite the wide availability of computer programs for matrix computations, it continues to be an active area of research and development. New applications, new algorithms, and improvements to old algorithms are constantly emerging. Presents the first book available on matrix algorithms implemented in real computer code Provides algorithms covered in three parts, the mathematical development of the algorithm using a simple example, the code implementation, and then numerical examples using the code Allows readers to gain a quick understanding of an algorithm by debugging or reading the source code Includes downloadable codes on an accompanying companion website, www.matrixalgorithmsinmatlab.com, that can be used in other software development

Many problems in the sciences and engineering can be rephrased as optimization problems on matrix search spaces endowed with a so-called manifold structure. This book shows how to exploit the special structure of such problems to develop efficient numerical algorithms. It places careful emphasis on both the numerical formulation of the algorithm and its differential geometric abstraction--illustrating how good algorithms draw equally from the insights of differential geometry, optimization, and numerical analysis. Two more theoretical chapters provide readers with the background in differential geometry necessary to algorithmic development. In the other chapters, several well-known optimization methods such as steepest descent and conjugate gradients are generalized to abstract manifolds. The book provides a generic development of each of these methods, building upon the material of the geometric chapters. It then guides readers through the calculations that turn these geometrically formulated methods into concrete numerical algorithms. The state-of-the-art algorithms given as examples are competitive with the best existing algorithms for a selection of eigenspace problems in numerical linear algebra. Optimization Algorithms on Matrix Manifolds offers techniques with broad applications in linear algebra, signal processing, data mining, computer vision, and statistical analysis. It can serve as a graduate-level textbook and will be of interest to applied mathematicians, engineers, and computer scientists.

This is the second volume in a projected five-volume survey of numerical linear algebra and matrix algorithms. It treats the numerical solution of dense and large-scale eigenvalue problems with an emphasis on algorithms and the theoretical background required to understand them. The notes and reference sections contain pointers to other methods along with historical comments. The book is divided into two parts: dense eigenproblems and large eigenproblems. The first part gives a full treatment of the widely used QR algorithm, which is then applied to the solution of generalized eigenproblems and the computation of the singular value decomposition. The second part treats Krylov sequence methods such as the Lanczos and Arnoldi algorithms and presents a new treatment of the Jacobi-Davidson method. These volumes are not intended to be encyclopedic, but provide the reader with the theoretical and practical background to read the research literature and implement or modify new algorithms.

This user-friendly, engaging textbook makes the material accessible to graduate students and new researchers who wish to study the rapidly exploding area of computations with structured matrices and polynomials. The book goes beyond research frontiers and, apart from very recent research articles, includes previously unpublished results.

Our Subjects and Objectives. This book is about algebraic and symbolic computation and numerical computing (with matrices and polynomials). It greatly extends the study of these topics presented in the celebrated books of the seventies, [AHU] and [BM] (these topics have been under-represented in [CLR], which is a highly successful extension and updating of [AHU] otherwise). Compared to [AHU] and [BM] our volume adds extensive material on parallel com putations with general matrices and polynomials, on the bit-complexity of arithmetic computations (including some recent techniques of data compres sion and the study of numerical approximation properties of polynomial and matrix algorithms), and on computations with Toeplitz matrices and other dense structured matrices. The latter subject should attract people working in numerous areas of application (in particular, coding, signal processing, control, algebraic computing and partial differential equations). The au thors' teaching experience at the Graduate Center of the City University of New York and at the University of Pisa suggests that the book may serve as a text for advanced graduate students in mathematics and computer science who have some knowledge of algorithm design and wish to enter the exciting area of algebraic and numerical computing. The potential readership may also include algorithm and software designers and researchers specializing in the design and analysis of algorithms, computational complexity, alge braic and symbolic computing, and numerical computation.