Analog Vlsi And Neural Systems

This volume contains the proceedings of a workshop on Analog Integrated Neural Systems held May 8, 1989, in connection with the International Symposium on Circuits and Systems. The presentations were chosen to encompass the entire range of topics currently under study in this exciting new discipline. Stringent acceptance requirements were placed on contributions: (1) each description was required to include detailed characterization of a working chip, and (2) each design was not to have been published previously. In several cases, the status of the project was not known until a few weeks before the meeting date. As a result, some of the most recent innovative work in the field was presented. Because this discipline is evolving rapidly, each project is very much a work in progress. Authors were asked to devote considerable attention to the shortcomings of their designs, as well as to the notable successes they achieved. In this way, other workers can now avoid stumbling into the same traps, and evolution can proceed more rapidly (and less painfully). The chapters in this volume are presented in the same order as the corresponding presentations at the workshop. The first two chapters are concerned with fmding solutions to complex optimization problems under a predefmed set of constraints. The first chapter reports what is, to the best of our knowledge, the first neural-chip design. In each case, the physics of the underlying electronic medium is used to represent a cost function in a natural way, using only nearest-neighbor connectivity.

A self-contained text, suitable for a broad audience. Presents basic concepts in electronics, transistor physics, and neurobiology for readers without backgrounds in those areas. Annotation copyrighted by Book News, Inc., Portland, OR

Adaptive Analog VLSI Neural Systems is the first practical book on neural networks learning chips and systems. It covers the entire process of implementing neural networks in VLSI chips, beginning with the crucial issues of learning algorithms in an analog framework and limited precision effects, and giving actual case studies of working systems. The approach is systems and applications oriented throughout, demonstrating the attractiveness of such an approach for applications such as adaptive pattern recognition and optical character recognition. Dr Jabri and his co-authors from AT&T Bell Laboratories, Bellcore and the University of Sydney provide a comprehensive introduction to VLSI neural networks suitable for research and development staff and advanced students.

This book brings together in one place important contributions and state-of-the-art research in the rapidly advancing area of analog VLSI neural networks. The book serves as an excellent reference, providing insights into some of the most important issues in analog VLSI neural networks research efforts.

Neural Information Processing and VLSI provides a unified treatment of this important subject for use in classrooms, industry, and research laboratories, in order to develop advanced artificial and biologically-inspired neural networks using compact analog and digital VLSI parallel processing techniques. Neural Information Processing and VLSI systematically presents various neural network paradigms, computing architectures, and the associated electronic/optical implementations using efficient VLSI design methodologies. Conventional digital machines cannot perform computationally-intensive tasks with satisfactory performance in such areas as intelligent perception, including visual and auditory signal processing, recognition, understanding, and logical reasoning (where the human being and even a small living animal can do a superb job). Recent research advances in artificial and biological neural networks have established an important foundation for high-performance information processing with more efficient use of computing resources. The secret lies in the design optimization at various levels of computing and communication of intelligent machines. Each neural network system consists of massively paralleled and distributed signal processors with every processor performing very simple operations, thus consuming little power. Large computational capabilities of these systems in the range of some hundred giga to several tera operations per second are derived from collectively parallel processing and efficient data routing, through well-structured interconnection networks. Deep-submicron very large-scale integration (VLSI) technologies can integrate tens of millions of transistors in a single silicon chip for complex signal processing and information manipulation. The book is suitable for those interested in efficient neurocomputing as well as those curious about neural network system applications. It has been especially prepared for use as a text for advanced undergraduate and first year graduate students, and is an excellent reference book for researchers and scientists working in the fields covered.

The early era of neural network hardware design (starting at 1985) was mainly technology driven. Designers used almost exclusively analog signal processing concepts for the recall mode. Learning was deemed not to cause a problem because the number of implementable synapses was still so low that the determination of weights and thresholds could be left to conventional computers. Instead, designers tried to directly map neural parallelity into hardware. The architectural concepts were accordingly simple and produced the so called interconnection problem which, in turn, made many engineers believe it could be solved by optical implementation in adequate fashion only. Furthermore, the inherent fault-tolerance and limited computation accuracy of neural networks were claimed to justify that little effort is to be spend on careful design, but most effort be put on technology issues. As a result, it was almost impossible to predict whether an electronic neural network would function in the way it was simulated to do. This limited the use of the first neuro-chips for further experimentation, not to mention that real-world applications called for much more synapses than could be implemented on a single chip at that time. Meanwhile matters have matured. It is recognized that isolated definition of the effort of analog multiplication, for instance, would be just as inappropriate on the part ofthe chip designer as determination of the weights by simulation, without allowing for the computing accuracy that can be achieved, on the part of the user.

Although it is now possible to integrate many millions of transistors on a single chip, traditional digital circuit technology is now reaching its limits, facing problems of cost and technical efficiency when scaled down to ever-smaller feature sizes. The analysis of biological neural systems, especially for visual processing, has allowed engineers to better understand how complex networks can effectively process large amounts of information, whilst dealing with difficult computational challenges. Analog and parallel processing are key characteristics of biological neural networks. Analog VLSI circuits using the same features can therefore be developed to emulate brain-style processing. Using standard CMOS technology, they can be cheaply manufactured, permitting efficient industrial and consumer applications in robotics and mobile electronics. This book explores the theory, design and implementation of analog VLSI circuits, inspired by visual motion processing in biological neural networks. Using a novel approach pioneered by the author himself, Stocker explains in detail the construction of a series of electronic chips, providing the reader with a valuable practical insight into the technology. Analog VLSI Circuits for the Perception of Visual Motion: analyses the computational problems in visual motion perception; examines the issue of optimization in analog networks through high level processes such as motion segmentation and selective attention; demonstrates network implementation in analog VLSI CMOS technology to provide computationally efficient devices; sets out measurements of final hardware implementation; illustrates the similarities of the presented circuits with the human visual motion perception system; includes an accompanying website with video clips of circuits under real-time visual conditions and additional supplementary material. With a complete review of all existing neuromorphic analog VLSI systems for visual motion sensing, Analog VLSI Circuits for the Perception of Visual Motion is a unique reference for advanced students in electrical engineering, artificial intelligence, robotics and computational neuroscience. It will also be useful for researchers, professionals, and electronics engineers working in the field.

Rapid advances in neural sciences and VLSI design technologies have provided an excellent means to boost the computational capability and efficiency of data and signal processing tasks by several orders of magnitude. With massively parallel processing capabilities, artificial neural networks can be used to solve many engineering and scientific problems. Due to the optimized data communication structure for artificial intelligence applications, a neurocomputer is considered as the most promising sixth-generation computing machine. Typical applica tions of artificial neural networks include associative memory, pattern classification, early vision processing, speech recognition, image data compression, and intelligent robot control. VLSI neural circuits play an important role in exploring and exploiting the rich properties of artificial neural networks by using pro grammable synapses and gain-adjustable neurons. Basic building blocks of the analog VLSI neural networks consist of operational amplifiers as electronic neurons and synthesized resistors as electronic synapses. The synapse weight information can be stored in the dynamically refreshed capacitors for medium-term storage or in the floating-gate of an EEPROM cell for long-term storage. The feedback path in the amplifier can continuously change the output neuron operation from the unity-gain configuration to a high-gain configuration. The adjustability of the vol tage gain in the output neurons allows the implementation of hardware annealing in analog VLSI neural chips to find optimal solutions very efficiently. Both supervised learning and unsupervised learning can be implemented by using the programmable neural chips.