Least Mean Square Adaptive Filters

Edited by the original inventor of the technology. Includes contributions by the foremost experts in the field. The only book to cover these topics together.

Adaptive filters are used in many diverse applications, appearing in everything from military instruments to cellphones and home appliances. Adaptive Filtering: Fundamentals of Least Mean Squares with MATLAB® covers the core concepts of this important field, focusing on a vital part of the statistical signal processing area—the least mean square (LMS) adaptive filter. This largely self-contained text: Discusses random variables, stochastic processes, vectors, matrices, determinants, discrete random signals, and probability distributions Explains how to find the eigenvalues and eigenvectors of a matrix and the properties of the error surfaces Explores the Wiener filter and its practical uses, details the steepest descent method, and develops the Newton’s algorithm Addresses the basics of the LMS adaptive filter algorithm, considers LMS adaptive filter variants, and provides numerous examples Delivers a concise introduction to MATLAB®, supplying problems, computer experiments, and more than 110 functions and script files Featuring robust appendices complete with mathematical tables and formulas, Adaptive Filtering: Fundamentals of Least Mean Squares with MATLAB® clearly describes the key principles of adaptive filtering and effectively demonstrates how to apply them to solve real-world problems.

Adaptive filters play an important role in the fields related to digital signal processing and communication, such as system identification, noise cancellation, channel equalization, and beamforming. In practical applications, the computational complexity of an adaptive filter is an important consideration. The Least Mean Square (LMS) algorithm is widely used because of its low computational complexity ($O(N)$) and simplicity in implementation. The least squares algorithms, such as Recursive Least Squares (RLS), Conjugate Gradient (CG), and Euclidean Direction Search (EDS), can converge faster and have lower steady-state mean square error (MSE) than LMS. However, their high computational complexity ($O(N^2)$) makes them unsuitable for many real-time applications. A well-known approach to controlling computational complexity is applying partial update (PU) method to adaptive filters. A partial update method can reduce the adaptive algorithm complexity by updating part of the weight vector instead of the entire vector or by updating part of the time. In the literature, there are only a few analyses of these partial update adaptive filter algorithms. Most analyses are based on partial update LMS and its variants. Only a few papers have addressed partial update RLS and Affine Projection (AP). Therefore, analyses for PU least-squares adaptive filter algorithms are necessary and meaningful. This monograph mostly focuses on the analyses of the partial update least-squares adaptive filter algorithms. Basic partial update methods are applied to adaptive filter algorithms including Least Squares CMA (LSCMA), EDS, and CG. The PU methods are also applied to CMA1-2 and NCMA to compare with the performance of the LSCMA. Mathematical derivation and performance analysis are provided including convergence condition, steady-state mean and mean-square performance for a time-invariant system. The steady-state mean and mean-square performance are also presented for a time-varying system. Computational complexity is calculated for each adaptive filter algorithm. Numerical examples are shown to compare the computational complexity of the PU adaptive filters with the full-update filters. Computer simulation examples, including system identification and channel equalization, are used to demonstrate the mathematical analysis and show the performance of PU adaptive filter algorithms. They also show the convergence performance of PU adaptive filters. The performance is compared between the original adaptive filter algorithms and different partial-update methods. The performance is also compared among similar PU least-squares adaptive filter algorithms, such as PU RLS, PU CG, and PU EDS. In addition to the generic applications of system identification and channel equalization, two special applications of using partial update adaptive filters are also presented. One application uses PU adaptive filters to detect Global System for Mobile Communication (GSM) signals in a local GSM system using the Open Base Transceiver Station (OpenBTS) and Asterisk Private Branch Exchange (PBX). The other application uses PU adaptive filters to do image compression in a system combining hyperspectral image compression and classification.

Edited by the original inventor of the technology. Includes contributions by the foremost experts in the field. The only book to cover these topics together.

In this book, the authors provide insights into the basics of adaptive filtering, which are particularly useful for students taking their first steps into this field. They start by studying the problem of minimum mean-square-error filtering, i.e., Wiener filtering. Then, they analyze iterative methods for solving the optimization problem, e.g., the Method of Steepest Descent. By proposing stochastic approximations, several basic adaptive algorithms are derived, including Least Mean Squares (LMS), Normalized Least Mean Squares (NLMS) and Sign-error algorithms. The authors provide a general framework to study the stability and steady-state performance of these algorithms. The affine Projection Algorithm (APA) which provides faster convergence at the expense of computational complexity (although fast implementations can be used) is also presented. In addition, the Least Squares (LS) method and its recursive version (RLS), including fast implementations are discussed. The book closes with the discussion of several topics of interest in the adaptive filtering field.

Adaptive Filter Theory, 4e, is ideal for courses in Adaptive Filters. Haykin examines both the mathematical theory behind various linear adaptive filters and the elements of supervised multilayer perceptrons. In its fourth edition, this highly successful book has been updated and refined to stay current with the field and develop concepts in as unified and accessible a manner as possible.

Adaptive filtering still receives attention in engineering as the use of the adaptive filter provides improved performance over the use of a fixed filter under the time-varying and unknown statistics environments. This application evolved communications, signal processing, seismology, mechanical design, and control engineering. The most popular optimization criterion in adaptive filtering is the well-known minimum mean square error (MMSE) criterion, which is, however, only optimal when the signals involved are Gaussian-distributed. Therefore, many "optimal solutions" under MMSE are not optimal. As an extension of the traditional MMSE, the minimum mean p-power error (MMPE) criterion has shown superior performance in many applications of adaptive filtering. This book aims to provide a comprehensive introduction of the MMPE and related adaptive filtering algorithms, which will become an important reference for researchers and practitioners in this application area. The book is geared to senior undergraduates with a basic understanding of linear algebra and statistics, graduate students, or practitioners with experience in adaptive signal processing. Key Features: Provides a systematic description of the MMPE criterion. Many adaptive filtering algorithms under MMPE, including linear and nonlinear filters, will be introduced. Extensive illustrative examples are included to demonstrate the results.