Bayesian Tensor Decomposition For Signal Processing And Machine Learning

This book presents recent advances of Bayesian inference in structured tensor decompositions. It explains how Bayesian modeling and inference lead to tuning-free tensor decomposition algorithms, which achieve state-of-the-art performances in many applications, including blind source separation; social network mining; image and video processing; array signal processing; and, wireless communications. The book begins with an introduction to the general topics of tensors and Bayesian theories. It then discusses probabilistic models of various structured tensor decompositions and their inference algorithms, with applications tailored for each tensor decomposition presented in the corresponding chapters. The book concludes by looking to the future, and areas where this research can be further developed. Bayesian Tensor Decomposition for Signal Processing and Machine Learning is suitable for postgraduates and researchers with interests in tensor data analytics and Bayesian methods.

Tensor is a natural representation for multi-dimensional data, and tensor computation can avoid possible multi-linear data structure loss in classical matrix computation-based data analysis. This book is intended to provide non-specialists an overall understanding of tensor computation and its applications in data analysis, and benefits researchers, engineers, and students with theoretical, computational, technical and experimental details. It presents a systematic and up-to-date overview of tensor decompositions from the engineer's point of view, and comprehensive coverage of tensor computation based data analysis techniques. In addition, some practical examples in machine learning, signal processing, data mining, computer vision, remote sensing, and biomedical engineering are also presented for easy understanding and implementation. These data analysis techniques may be further applied in other applications on neuroscience, communication, psychometrics, chemometrics, biometrics, quantum physics, quantum chemistry, etc. The discussion begins with basic coverage of notations, preliminary operations in tensor computations, main tensor decompositions and their properties. Based on them, a series of tensor-based data analysis techniques are presented as the tensor extensions of their classical matrix counterparts, including tensor dictionary learning, low rank tensor recovery, tensor completion, coupled tensor analysis, robust principal tensor component analysis, tensor regression, logistical tensor regression, support tensor machine, multilinear discriminate analysis, tensor subspace clustering, tensor-based deep learning, tensor graphical model and tensor sketch. The discussion also includes a number of typical applications with experimental results, such as image reconstruction, image enhancement, data fusion, signal recovery, recommendation system, knowledge graph acquisition, traffic flow prediction, link prediction, environmental prediction, weather forecasting, background extraction, human pose estimation, cognitive state classification from fMRI, infrared small target detection, heterogeneous information networks clustering, multi-view image clustering, and deep neural network compression.

Tensors for Data Processing: Theory, Methods and Applications presents both classical and state-of-the-art methods on tensor computation for data processing, covering computation theories, processing methods, computing and engineering applications, with an emphasis on techniques for data processing. This reference is ideal for students, researchers and industry developers who want to understand and use tensor-based data processing theories and methods. As a higher-order generalization of a matrix, tensor-based processing can avoid multi-linear data structure loss that occurs in classical matrix-based data processing methods. This move from matrix to tensors is beneficial for many diverse application areas, including signal processing, computer science, acoustics, neuroscience, communication, medical engineering, seismology, psychometric, chemometrics, biometric, quantum physics and quantum chemistry. Provides a complete reference on classical and state-of-the-art tensor-based methods for data processing Includes a wide range of applications from different disciplines Gives guidance for their application

Machine Learning: A Bayesian and Optimization Perspective, 2nd edition, gives a unified perspective on machine learning by covering both pillars of supervised learning, namely regression and classification. The book starts with the basics, including mean square, least squares and maximum likelihood methods, ridge regression, Bayesian decision theory classification, logistic regression, and decision trees. It then progresses to more recent techniques, covering sparse modelling methods, learning in reproducing kernel Hilbert spaces and support vector machines, Bayesian inference with a focus on the EM algorithm and its approximate inference variational versions, Monte Carlo methods, probabilistic graphical models focusing on Bayesian networks, hidden Markov models and particle filtering. Dimensionality reduction and latent variables modelling are also considered in depth. This palette of techniques concludes with an extended chapter on neural networks and deep learning architectures. The book also covers the fundamentals of statistical parameter estimation, Wiener and Kalman filtering, convexity and convex optimization, including a chapter on stochastic approximation and the gradient descent family of algorithms, presenting related online learning techniques as well as concepts and algorithmic versions for distributed optimization. Focusing on the physical reasoning behind the mathematics, without sacrificing rigor, all the various methods and techniques are explained in depth, supported by examples and problems, giving an invaluable resource to the student and researcher for understanding and applying machine learning concepts. Most of the chapters include typical case studies and computer exercises, both in MATLAB and Python. The chapters are written to be as self-contained as possible, making the text suitable for different courses: pattern recognition, statistical/adaptive signal processing, statistical/Bayesian learning, as well as courses on sparse modeling, deep learning, and probabilistic graphical models. New to this edition: Complete re-write of the chapter on Neural Networks and Deep Learning to reflect the latest advances since the 1st edition. The chapter, starting from the basic perceptron and feed-forward neural networks concepts, now presents an in depth treatment of deep networks, including recent optimization algorithms, batch normalization, regularization techniques such as the dropout method, convolutional neural networks, recurrent neural networks, attention mechanisms, adversarial examples and training, capsule networks and generative architectures, such as restricted Boltzman machines (RBMs), variational autoencoders and generative adversarial networks (GANs). Expanded treatment of Bayesian learning to include nonparametric Bayesian methods, with a focus on the Chinese restaurant and the Indian buffet processes. Presents the physical reasoning, mathematical modeling and algorithmic implementation of each method Updates on the latest trends, including sparsity, convex analysis and optimization, online distributed algorithms, learning in RKH spaces, Bayesian inference, graphical and hidden Markov models, particle filtering, deep learning, dictionary learning and latent variables modeling Provides case studies on a variety of topics, including protein folding prediction, optical character recognition, text authorship identification, fMRI data analysis, change point detection, hyperspectral image unmixing, target localization, and more

Treating VB approximation in signal processing, this monograph is for academic and industrial research groups in signal processing, data analysis, machine learning and identification. It reviews distributional approximation, showing that tractable algorithms for parametric model identification can be generated in off-line and on-line contexts.

Signal Processing and Machine Learning Theory, authored by world-leading experts, reviews the principles, methods and techniques of essential and advanced signal processing theory. These theories and tools are the driving engines of many current and emerging research topics and technologies, such as machine learning, autonomous vehicles, the internet of things, future wireless communications, medical imaging, etc. Provides quick tutorial reviews of important and emerging topics of research in signal processing-based tools Presents core principles in signal processing theory and shows their applications Discusses some emerging signal processing tools applied in machine learning methods References content on core principles, technologies, algorithms and applications Includes references to journal articles and other literature on which to build further, more specific, and detailed knowledge

Describes in detail the fundamental mathematics and algorithms of machine learning (an example of artificial intelligence) and signal processing, two of the most important and exciting technologies in the modern information economy. Builds up concepts gradually so that the ideas and algorithms can be implemented in practical software applications.

Source Separation and Machine Learning presents the fundamentals in adaptive learning algorithms for Blind Source Separation (BSS) and emphasizes the importance of machine learning perspectives. It illustrates how BSS problems are tackled through adaptive learning algorithms and model-based approaches using the latest information on mixture signals to build a BSS model that is seen as a statistical model for a whole system. Looking at different models, including independent component analysis (ICA), nonnegative matrix factorization (NMF), nonnegative tensor factorization (NTF), and deep neural network (DNN), the book addresses how they have evolved to deal with multichannel and single-channel source separation. Emphasizes the modern model-based Blind Source Separation (BSS) which closely connects the latest research topics of BSS and Machine Learning Includes coverage of Bayesian learning, sparse learning, online learning, discriminative learning and deep learning Presents a number of case studies of model-based BSS (categorizing them into four modern models - ICA, NMF, NTF and DNN), using a variety of learning algorithms that provide solutions for the construction of BSS systems