An Information Theoretic Approach To Neural Computing

A detailed formulation of neural networks from the information-theoretic viewpoint. The authors show how this perspective provides new insights into the design theory of neural networks. In particular they demonstrate how these methods may be applied to the topics of supervised and unsupervised learning, including feature extraction, linear and non-linear independent component analysis, and Boltzmann machines. Readers are assumed to have a basic understanding of neural networks, but all the relevant concepts from information theory are carefully introduced and explained. Consequently, readers from varied scientific disciplines, notably cognitive scientists, engineers, physicists, statisticians, and computer scientists, will find this an extremely valuable introduction to this topic.

Information theoretics vis-a-vis neural networks generally embodies parametric entities and conceptual bases pertinent to memory considerations and information storage, information-theoretic based cost-functions, and neurocybernetics and self-organization. Existing studies only sparsely cover the entropy and/or cybernetic aspects of neural information. Information-Theoretic Aspects of Neural Networks cohesively explores this burgeoning discipline, covering topics such as: Shannon information and information dynamics neural complexity as an information processing system memory and information storage in the interconnected neural web extremum (maximum and minimum) information entropy neural network training non-conventional, statistical distance-measures for neural network optimizations symmetric and asymmetric characteristics of information-theoretic error-metrics algorithmic complexity based representation of neural information-theoretic parameters genetic algorithms versus neural information dynamics of neurocybernetics viewed in the information-theoretic plane nonlinear, information-theoretic transfer function of the neural cellular units statistical mechanics, neural networks, and information theory semiotic framework of neural information processing and neural information flow fuzzy information and neural networks neural dynamics conceived through fuzzy information parameters neural information flow dynamics informatics of neural stochastic resonance Information-Theoretic Aspects of Neural Networks acts as an exceptional resource for engineers, scientists, and computer scientists working in the field of artificial neural networks as well as biologists applying the concepts of communication theory and protocols to the functioning of the brain. The information in this book explores new avenues in the field and creates a common platform for analyzing the neural complex as well as artificial neural networks.

In order to develop new types of information media and technology, it is essential to model complex and flexible information processing in living systems. This book presents a new approach to modeling complex information processing in living systems. Traditional information-theoretic methods in neural networks are unified in one framework, i.e. α-entropy. This new approach will enable information systems such as computers to imitate and simulate human complex behavior and to uncover the deepest secrets of the human mind. Contents: Information in Neural NetworksInformation MinimizationInformation MaximizationConstrained Information MaximizationNeural Feature DetectorsInformation Maximization and MinimizationInformation ControllerInformation Control by α-EntropyIntegrated Information Processing Systems Readership: Students and researchers in artificial intelligence and neural networks. Keywords:

This book is the first cohesive treatment of ITL algorithms to adapt linear or nonlinear learning machines both in supervised and unsupervised paradigms. It compares the performance of ITL algorithms with the second order counterparts in many applications.

Premiering in 1990 in Antibes, France, the European Conference on Computer Vision, ECCV, has been held biennially at venues all around Europe. These conferences have been very successful, making ECCV a major event to the computer vision community. ECCV 2002 was the seventh in the series. The privilege of organizing it was shared by three universities: The IT University of Copenhagen, the University of Copenhagen, and Lund University, with the conference venue in Copenhagen. These universities lie ̈ geographically close in the vivid Oresund region, which lies partly in Denmark and partly in Sweden, with the newly built bridge (opened summer 2000) crossing the sound that formerly divided the countries. We are very happy to report that this year’s conference attracted more papers than ever before, with around 600 submissions. Still, together with the conference board, we decided to keep the tradition of holding ECCV as a single track conference. Each paper was anonymously refereed by three different reviewers. For the nal selection, for the rst time for ECCV, a system with area chairs was used. These met with the program chairsinLundfortwodaysinFebruary2002toselectwhatbecame45oralpresentations and 181 posters.Also at this meeting the selection was made without knowledge of the authors’identity.

Recently, criterion functions based on information theoretic measures (entropy, mutual information, information divergence) have attracted attention and become an emerging area of study in signal processing and system identification domain. This book presents a systematic framework for system identification and information processing, investigating system identification from an information theory point of view. The book is divided into six chapters, which cover the information needed to understand the theory and application of system parameter identification. The authors’ research provides a base for the book, but it incorporates the results from the latest international research publications. Named a 2013 Notable Computer Book for Information Systems by Computing Reviews One of the first books to present system parameter identification with information theoretic criteria so readers can track the latest developments Contains numerous illustrative examples to help the reader grasp basic methods

The refereed proceedings of the Joint International Conference on Artificial Neural Networks and International Conference on Neural Information Processing, ICANN/ICONIP 2003, held in Istanbul, Turkey, in June 2003. The 138 revised full papers were carefully reviewed and selected from 346 submissions. The papers are organized in topical sections on learning algorithms, support vector machine and kernel methods, statistical data analysis, pattern recognition, vision, speech recognition, robotics and control, signal processing, time-series prediction, intelligent systems, neural network hardware, cognitive science, computational neuroscience, context aware systems, complex-valued neural networks, emotion recognition, and applications in bioinformatics.

Table of contents