Feedforward Neural Network Methodology

This decade has seen an explosive growth in computational speed and memory and a rapid enrichment in our understanding of artificial neural networks. These two factors provide systems engineers and statisticians with the ability to build models of physical, economic, and information-based time series and signals. This book provides a thorough and coherent introduction to the mathematical properties of feedforward neural networks and to the intensive methodology which has enabled their highly successful application to complex problems.

Feed-Forward Neural Networks: Vector Decomposition Analysis, Modelling and Analog Implementation presents a novel method for the mathematical analysis of neural networks that learn according to the back-propagation algorithm. The book also discusses some other recent alternative algorithms for hardware implemented perception-like neural networks. The method permits a simple analysis of the learning behaviour of neural networks, allowing specifications for their building blocks to be readily obtained. Starting with the derivation of a specification and ending with its hardware implementation, analog hard-wired, feed-forward neural networks with on-chip back-propagation learning are designed in their entirety. On-chip learning is necessary in circumstances where fixed weight configurations cannot be used. It is also useful for the elimination of most mis-matches and parameter tolerances that occur in hard-wired neural network chips. Fully analog neural networks have several advantages over other implementations: low chip area, low power consumption, and high speed operation. Feed-Forward Neural Networks is an excellent source of reference and may be used as a text for advanced courses.

Neural networks represent a powerful data processing technique that has reached maturity and broad application. When clearly understood and appropriately used, they are a mandatory component in the toolbox of any engineer who wants make the best use of the available data, in order to build models, make predictions, mine data, recognize shapes or signals, etc. Ranging from theoretical foundations to real-life applications, this book is intended to provide engineers and researchers with clear methodologies for taking advantage of neural networks in industrial, financial or banking applications, many instances of which are presented in the book. For the benefit of readers wishing to gain deeper knowledge of the topics, the book features appendices that provide theoretical details for greater insight, and algorithmic details for efficient programming and implementation. The chapters have been written by experts and edited to present a coherent and comprehensive, yet not redundant, practically oriented introduction.

Artificial neural networks are nonlinear mapping systems whose structure is loosely based on principles observed in the nervous systems of humans and animals. The basic idea is that massive systems of simple units linked together in appropriate ways can generate many complex and interesting behaviors. This book focuses on the subset of feedforward artificial neural networks called multilayer perceptrons (MLP). These are the mostly widely used neural networks, with applications as diverse as finance (forecasting), manufacturing (process control), and science (speech and image recognition). This book presents an extensive and practical overview of almost every aspect of MLP methodology, progressing from an initial discussion of what MLPs are and how they might be used to an in-depth examination of technical factors affecting performance. The book can be used as a tool kit by readers interested in applying networks to specific problems, yet it also presents theory and references outlining the last ten years of MLP research.

This book is part of a three volume set that constitutes the refereed proceedings of the 4th International Symposium on Neural Networks, ISNN 2007, held in Nanjing, China in June 2007. Coverage includes neural networks for control applications, robotics, data mining and feature extraction, chaos and synchronization, support vector machines, fault diagnosis/detection, image/video processing, and applications of neural networks.

What Is Feedforward Neural Networks A feedforward neural network, often known as a FNN, is a type of artificial neural network that does not have connections that form a cycle between its nodes. Therefore, it is distinct from its offspring, which are known as recurrent neural networks. How You Will Benefit (I) Insights, and validations about the following topics: Chapter 1: Feedforward neural network Chapter 2: Artificial neural network Chapter 3: Perceptron Chapter 4: Artificial neuron Chapter 5: Multilayer perceptron Chapter 6: Delta rule Chapter 7: Backpropagation Chapter 8: Types of artificial neural networks Chapter 9: Learning rule Chapter 10: Mathematics of artificial neural networks (II) Answering the public top questions about feedforward neural networks. (III) Real world examples for the usage of feedforward neural networks in many fields. Who This Book Is For Professionals, undergraduate and graduate students, enthusiasts, hobbyists, and those who want to go beyond basic knowledge or information for any kind of feedforward neural networks. What Is Artificial Intelligence Series The Artificial Intelligence eBook series provides comprehensive coverage in over 200 topics. Each ebook covers a specific Artificial Intelligence topic in depth, written by experts in the field. The series aims to give readers a thorough understanding of the concepts, techniques, history and applications of artificial intelligence. Topics covered include machine learning, deep learning, neural networks, computer vision, natural language processing, robotics, ethics and more. The ebooks are written for professionals, students, and anyone interested in learning about the latest developments in this rapidly advancing field. The Artificial Intelligence eBook series provides an in-depth yet accessible exploration, from the fundamental concepts to the state-of-the-art research. With over 200 volumes, readers gain a thorough grounding in all aspects of Artificial Intelligence. The ebooks are designed to build knowledge systematically, with later volumes building on the foundations laid by earlier ones. This comprehensive series is an indispensable resource for anyone seeking to develop expertise in artificial intelligence.

For the first time, this book sets forth the concept and model for a process neural network. You’ll discover how a process neural network expands the mapping relationship between the input and output of traditional neural networks and greatly enhances the expression capability of artificial neural networks. Detailed illustrations help you visualize information processing flow and the mapping relationship between inputs and outputs.

In order to facilitate complexity optimization in feedforward networks, several inte- grated growing and pruning algorithms are developed. First, a growing scheme is reviewed which iteratively adds new hidden units to full-trained networks. Then, a non-heuristic one-pass pruning technique is reviewed, which utilizes orthogonal least squares. Based upon pruning, a one-pass approach is developed for producing the validation error versus network size curve. Then, a combined approach is devised in which grown networks are pruned. As a result, the hidden units are ordered according to their usefulness, and less useful units are eliminated. In several examples, it is shown that networks designed using the integrated growing and pruning method have less training and validation error. This combined method exhibits reduced sensitivity to the choice of the initial weights and produces an almost monotonic error versus network size curve. Starting from the strict interpolation equations for multivariate polynomials, an upper bound is developed for the number of patterns that can be memorized by a non-linear feedforward network. A straightforward proof by contradiction is presented for the upper bound. It is shown that the hidden activations do not have to be analytic. Networks, trained by conjugate gradient, are used to demonstrate the tightness of the bound for random patterns. The theoretical results agree closely to the simulations on two class problems solved by support vector machines. We model large classifiers like Support Vector Machines (SVMs) by smaller networks in order to decrease the computational cost. The key idea is to generate additional training patterns using a trained SVM and use these additional patterns along with the original training patterns to train a much smaller neural network. Results shown verify the validity of the technique and the method used to generate additional patterns. We also generalize this idea and prove that any learning machine can be used to generate additional patterns and in turn train any other machine to improve its performance.