Tensor Network Contractions

Tensor network is a fundamental mathematical tool with a huge range of applications in physics, such as condensed matter physics, statistic physics, high energy physics, and quantum information sciences. This open access book aims to explain the tensor network contraction approaches in a systematic way, from the basic definitions to the important applications. This book is also useful to those who apply tensor networks in areas beyond physics, such as machine learning and the big-data analysis. Tensor network originates from the numerical renormalization group approach proposed by K. G. Wilson in 1975. Through a rapid development in the last two decades, tensor network has become a powerful numerical tool that can efficiently simulate a wide range of scientific problems, with particular success in quantum many-body physics. Varieties of tensor network algorithms have been proposed for different problems. However, the connections among different algorithms are not well discussed or reviewed. To fill this gap, this book explains the fundamental concepts and basic ideas that connect and/or unify different strategies of the tensor network contraction algorithms. In addition, some of the recent progresses in dealing with tensor decomposition techniques and quantum simulations are also represented in this book to help the readers to better understand tensor network. This open access book is intended for graduated students, but can also be used as a professional book for researchers in the related fields. To understand most of the contents in the book, only basic knowledge of quantum mechanics and linear algebra is required. In order to fully understand some advanced parts, the reader will need to be familiar with notion of condensed matter physics and quantum information, that however are not necessary to understand the main parts of the book. This book is a good source for non-specialists on quantum physics to understand tensor network algorithms and the related mathematics.

Tensor network is a fundamental mathematical tool with a huge range of applications in physics, such as condensed matter physics, statistic physics, high energy physics, and quantum information sciences. This open access book aims to explain the tensor network contraction approaches in a systematic way, from the basic definitions to the important applications. This book is also useful to those who apply tensor networks in areas beyond physics, such as machine learning and the big-data analysis. Tensor network originates from the numerical renormalization group approach proposed by K. G. Wilson in 1975. Through a rapid development in the last two decades, tensor network has become a powerful numerical tool that can efficiently simulate a wide range of scientific problems, with particular success in quantum many-body physics. Varieties of tensor network algorithms have been proposed for different problems. However, the connections among different algorithms are not well discussed or reviewed. To fill this gap, this book explains the fundamental concepts and basic ideas that connect and/or unify different strategies of the tensor network contraction algorithms. In addition, some of the recent progresses in dealing with tensor decomposition techniques and quantum simulations are also represented in this book to help the readers to better understand tensor network. This open access book is intended for graduated students, but can also be used as a professional book for researchers in the related fields. To understand most of the contents in the book, only basic knowledge of quantum mechanics and linear algebra is required. In order to fully understand some advanced parts, the reader will need to be familiar with notion of condensed matter physics and quantum information, that however are not necessary to understand the main parts of the book. This book is a good source for non-specialists on quantum physics to understand tensor network algorithms and the related mathematics. This work was published by Saint Philip Street Press pursuant to a Creative Commons license permitting commercial use. All rights not granted by the work's license are retained by the author or authors.

This monograph builds on Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions by discussing tensor network models for super-compressed higher-order representation of data/parameters and cost functions, together with an outline of their applications in machine learning and data analytics. A particular emphasis is on elucidating, through graphical illustrations, that by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volume of data/parameters, thereby alleviating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification, generalized eigenvalue decomposition and in the optimization of deep neural networks. The monograph focuses on tensor train (TT) and Hierarchical Tucker (HT) decompositions and their extensions, and on demonstrating the ability of tensor networks to provide scalable solutions for a variety of otherwise intractable large-scale optimization problems. Tensor Networks for Dimensionality Reduction and Large-scale Optimization Parts 1 and 2 can be used as stand-alone texts, or together as a comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions. See also: Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions. ISBN 978-1-68083-222-8

This volume of lecture notes briefly introduces the basic concepts needed in any computational physics course: software and hardware, programming skills, linear algebra, and differential calculus. It then presents more advanced numerical methods to tackle the quantum many-body problem: it reviews the numerical renormalization group and then focuses on tensor network methods, from basic concepts to gauge invariant ones. Finally, in the last part, the author presents some applications of tensor network methods to equilibrium and out-of-equilibrium correlated quantum matter. The book can be used for a graduate computational physics course. After successfully completing such a course, a student should be able to write a tensor network program and can begin to explore the physics of many-body quantum systems. The book can also serve as a reference for researchers working or starting out in the field.

Modern applications in engineering and data science are increasingly based on multidimensional data of exceedingly high volume, variety, and structural richness. However, standard machine learning algorithms typically scale exponentially with data volume and complexity of cross-modal couplings - the so called curse of dimensionality - which is prohibitive to the analysis of large-scale, multi-modal and multi-relational datasets. Given that such data are often efficiently represented as multiway arrays or tensors, it is therefore timely and valuable for the multidisciplinary machine learning and data analytic communities to review low-rank tensor decompositions and tensor networks as emerging tools for dimensionality reduction and large scale optimization problems. Our particular emphasis is on elucidating that, by virtue of the underlying low-rank approximations, tensor networks have the ability to alleviate the curse of dimensionality in a number of applied areas. In Part 1 of this monograph we provide innovative solutions to low-rank tensor network decompositions and easy to interpret graphical representations of the mathematical operations on tensor networks. Such a conceptual insight allows for seamless migration of ideas from the flat-view matrices to tensor network operations and vice versa, and provides a platform for further developments, practical applications, and non-Euclidean extensions. It also permits the introduction of various tensor network operations without an explicit notion of mathematical expressions, which may be beneficial for many research communities that do not directly rely on multilinear algebra. Our focus is on the Tucker and tensor train (TT) decompositions and their extensions, and on demonstrating the ability of tensor networks to provide linearly or even super-linearly (e.g., logarithmically) scalable solutions, as illustrated in detail in Part 2 of this monograph.

Tensors are ubiquitous in the sciences. The geometry of tensors is both a powerful tool for extracting information from data sets, and a beautiful subject in its own right. This book has three intended uses: a classroom textbook, a reference work for researchers in the sciences, and an account of classical and modern results in (aspects of) the theory that will be of interest to researchers in geometry. For classroom use, there is a modern introduction to multilinear algebra and to the geometry and representation theory needed to study tensors, including a large number of exercises. For researchers in the sciences, there is information on tensors in table format for easy reference and a summary of the state of the art in elementary language. This is the first book containing many classical results regarding tensors. Particular applications treated in the book include the complexity of matrix multiplication, P versus NP, signal processing, phylogenetics, and algebraic statistics. For geometers, there is material on secant varieties, G-varieties, spaces with finitely many orbits and how these objects arise in applications, discussions of numerous open questions in geometry arising in applications, and expositions of advanced topics such as the proof of the Alexander-Hirschowitz theorem and of the Weyman-Kempf method for computing syzygies.

This book is a tribute to Professor Jacek Żurada, who is best known for his contributions to computational intelligence and knowledge-based neurocomputing. It is dedicated to Professor Jacek Żurada, Full Professor at the Computational Intelligence Laboratory, Department of Electrical and Computer Engineering, J.B. Speed School of Engineering, University of Louisville, Kentucky, USA, as a token of appreciation for his scientific and scholarly achievements, and for his longstanding service to many communities, notably the computational intelligence community, in particular neural networks, machine learning, data analyses and data mining, but also the fuzzy logic and evolutionary computation communities, to name but a few. At the same time, the book recognizes and honors Professor Żurada’s dedication and service to many scientific, scholarly and professional societies, especially the IEEE (Institute of Electrical and Electronics Engineers), the world’s largest professional technical professional organization dedicated to advancing science and technology in a broad spectrum of areas and fields. The volume is divided into five major parts, the first of which addresses theoretic, algorithmic and implementation problems related to the intelligent use of data in the sense of how to derive practically useful information and knowledge from data. In turn, Part 2 is devoted to various aspects of neural networks and connectionist systems. Part 3 deals with essential tools and techniques for intelligent technologies in systems modeling and Part 4 focuses on intelligent technologies in decision-making, optimization and control, while Part 5 explores the applications of intelligent technologies.