Adaptive Filtering Prediction And Control

This unified survey focuses on linear discrete-time systems and explores natural extensions to nonlinear systems. It emphasizes discrete-time systems, summarizing theoretical and practical aspects of a large class of adaptive algorithms. 1984 edition.

The area of adaptive systems, which encompasses recursive identification, adaptive control, filtering, and signal processing, has been one of the most active areas of the past decade. Since adaptive controllers are fundamentally nonlinear controllers which are applied to nominally linear, possibly stochastic and time-varying systems, their theoretical analysis is usually very difficult. Nevertheless, over the past decade much fundamental progress has been made on some key questions concerning their stability, convergence, performance, and robustness. Moreover, adaptive controllers have been successfully employed in numerous practical applications, and have even entered the marketplace.

This volume surveys the major results and techniques of analysis in the field of adaptive control. Focusing on linear, continuous time, single-input, single-output systems, the authors offer a clear, conceptual presentation of adaptive methods, enabling a critical evaluation of these techniques and suggesting avenues of further development. 1989 edition.

Online learning from a signal processing perspective There is increased interest in kernel learning algorithms in neural networks and a growing need for nonlinear adaptive algorithms in advanced signal processing, communications, and controls. Kernel Adaptive Filtering is the first book to present a comprehensive, unifying introduction to online learning algorithms in reproducing kernel Hilbert spaces. Based on research being conducted in the Computational Neuro-Engineering Laboratory at the University of Florida and in the Cognitive Systems Laboratory at McMaster University, Ontario, Canada, this unique resource elevates the adaptive filtering theory to a new level, presenting a new design methodology of nonlinear adaptive filters. Covers the kernel least mean squares algorithm, kernel affine projection algorithms, the kernel recursive least squares algorithm, the theory of Gaussian process regression, and the extended kernel recursive least squares algorithm Presents a powerful model-selection method called maximum marginal likelihood Addresses the principal bottleneck of kernel adaptive filters—their growing structure Features twelve computer-oriented experiments to reinforce the concepts, with MATLAB codes downloadable from the authors' Web site Concludes each chapter with a summary of the state of the art and potential future directions for original research Kernel Adaptive Filtering is ideal for engineers, computer scientists, and graduate students interested in nonlinear adaptive systems for online applications (applications where the data stream arrives one sample at a time and incremental optimal solutions are desirable). It is also a useful guide for those who look for nonlinear adaptive filtering methodologies to solve practical problems.

This book was written in response to the growing demand for a text that provides a unified treatment of linear and nonlinear complex valued adaptive filters, and methods for the processing of general complex signals (circular and noncircular). It brings together adaptive filtering algorithms for feedforward (transversal) and feedback architectures and the recent developments in the statistics of complex variable, under the powerful frameworks of CR (Wirtinger) calculus and augmented complex statistics. This offers a number of theoretical performance gains, which is illustrated on both stochastic gradient algorithms, such as the augmented complex least mean square (ACLMS), and those based on Kalman filters. This work is supported by a number of simulations using synthetic and real world data, including the noncircular and intermittent radar and wind signals.

Adaptive filtering is useful in any application where the signals or the modeled system vary over time. The configuration of the system and, in particular, the position where the adaptive processor is placed generate different areas or application fields such as: prediction, system identification and modeling, equalization, cancellation of interference, etc. which are very important in many disciplines such as control systems, communications, signal processing, acoustics, voice, sound and image, etc. The book consists of noise and echo cancellation, medical applications, communications systems and others hardly joined by their heterogeneity. Each application is a case study with rigor that shows weakness/strength of the method used, assesses its suitability and suggests new forms and areas of use. The problems are becoming increasingly complex and applications must be adapted to solve them. The adaptive filters have proven to be useful in these environments of multiple input/output, variant-time behaviors, and long and complex transfer functions effectively, but fundamentally they still have to evolve. This book is a demonstration of this and a small illustration of everything that is to come.

Stochastic convergence theory is reviewed in this text including 33 fundamental martingale and convergence theorems. The book unifies identification theory; adaptive filtering; control and decision, and time series analysis. Examples of practical microcomputer-based applications are included.

This thesis examines the basic asymptotic properties of various stochastic adaptive systems for identification, filtering, prediction and control. These include the convergence of long-term averages of signals of interest (self-optimality), the convergence of adaptive filters or controllers (self-tuning property), the convergence of parameter estimates, and the rates of convergence. This thesis divides itself naturally into two parts. The first part considers identification, adaptive prediction and control based on the ARMAX model, while the second part considers general stochastic parallel model adaptation problems, which include output error identification, adaptive IIR filtering, adaptive noise cancelling, and adaptive feedforward control with or without input contamination. In the first part, the use of a generalized certainty equivalence approach in which the estimates of disturbance as well as parameters are utilized is proposed. Based on this, the self-optimality of adaptive minimum variance prediction and model reference adaptive control is established for systems with general delay and colored noise. Both direct and indirect approaches based on the extended least squares as well as the stochastic gradient algorithms are considered. For the direct approach, it is shown that interlacing is not necessary for convergence, thus resolving this long-standing open problem. Concerning the self-tuning property, it is established that self-optimality in the mean square sense, in general, implies self-tuning, by exhibiting the convergence of the parameter estimates to the null space of a certain covariance matrix, and by characterizing this null space. It is found that adaptive minimum variance regulators self-tune because of the "internal excitation" due to the plant disturbance alone. Finally, the exact order of external excitation required for the parameter estimates to converge to the true parameter is determined. In the second part of the thesis, the convergence of several parallel model adaptation schemes in the presence of nonstationary colored noise is established. A special case of our results resolves the long-standing problem of the convergence and unbiasedness of the output error identification scheme in the presence of colored noise. We also develop a simple general technique for analyzing the strong consistency of parameter estimation with projection. Of pedagogical interest is the deterministic reduction viewpoint we adopt in which all relevant properties of stochastically modeled disturbances are characterized deterministically by some long-term average properties. Readers more familiar with deterministic theory may well find this viewpoint to be more enlightening with respect to understanding the goals and results of stochastic adaptive system theory.