Self Organizing Neural Networks

The Self-Organizing Map (SOM) is one of the most frequently used architectures for unsupervised artificial neural networks. Introduced by Teuvo Kohonen in the 1980s, SOMs have been developed as a very powerful method for visualization and unsupervised classification tasks by an active and innovative community of interna tional researchers. A number of extensions and modifications have been developed during the last two decades. The reason is surely not that the original algorithm was imperfect or inad equate. It is rather the universal applicability and easy handling of the SOM. Com pared to many other network paradigms, only a few parameters need to be arranged and thus also for a beginner the network leads to useful and reliable results. Never theless there is scope for improvements and sophisticated new developments as this book impressively demonstrates. The number of published applications utilizing the SOM appears to be unending. As the title of this book indicates, the reader will benefit from some of the latest the oretical developments and will become acquainted with a number of challenging real-world applications. Our aim in producing this book has been to provide an up to-date treatment of the field of self-organizing neural networks, which will be ac cessible to researchers, practitioners and graduated students from diverse disciplines in academics and industry. We are very grateful to the father of the SOMs, Professor Teuvo Kohonen for sup porting this book and contributing the first chapter.

The book we have at hand is the fourth monograph I wrote for Springer Verlag. The previous one named "Self-Organization and Associative Mem ory" (Springer Series in Information Sciences, Volume 8) came out in 1984. Since then the self-organizing neural-network algorithms called SOM and LVQ have become very popular, as can be seen from the many works re viewed in Chap. 9. The new results obtained in the past ten years or so have warranted a new monograph. Over these years I have also answered lots of questions; they have influenced the contents of the present book. I hope it would be of some interest and help to the readers if I now first very briefly describe the various phases that led to my present SOM research, and the reasons underlying each new step. I became interested in neural networks around 1960, but could not in terrupt my graduate studies in physics. After I was appointed Professor of Electronics in 1965, it still took some years to organize teaching at the uni versity. In 1968 - 69 I was on leave at the University of Washington, and D. Gabor had just published his convolution-correlation model of autoasso ciative memory. I noticed immediately that there was something not quite right about it: the capacity was very poor and the inherent noise and crosstalk were intolerable. In 1970 I therefore sugge~ted the auto associative correlation matrix memory model, at the same time as J.A. Anderson and K. Nakano.

The conception of fresh ideas and the development of new techniques for Blind Source Separation and Independent Component Analysis have been rapid in recent years. It is also encouraging, from the perspective of the many scientists involved in this fascinating area of research, to witness the growing list of successful applications of these methods to a diverse range of practical everyday problems. This growth has been due, in part, to the number of promising young and enthusiastic researchers who have committed their efforts to expanding the current body of knowledge within this field of research. The author of this book is among one of their number. I trust that the present book by Dr. Mark Girolami will provide a rapid and effective means of communicating some of these new ideas to a wide international audience and that in turn this will expand further the growth of knowledge. In my opinion this book makes an important contribution to the theory of Independent Component Analysis and Blind Source Separation. This opens a range of exciting methods, techniques and algorithms for applied researchers and practitioner engineers, especially from the perspective of artificial neural networks and information theory. It has been interesting to see how rapidly the scientific literature in this area has grown.

This book provides an overview of self-organizing map formation, including recent developments. Self-organizing maps form a branch of unsupervised learning, which is the study of what can be determined about the statistical properties of input data without explicit feedback from a teacher. The articles are drawn from the journal Neural Computation.The book consists of five sections. The first section looks at attempts to model the organization of cortical maps and at the theory and applications of the related artificial neural network algorithms. The second section analyzes topographic maps and their formation via objective functions. The third section discusses cortical maps of stimulus features. The fourth section discusses self-organizing maps for unsupervised data analysis. The fifth section discusses extensions of self-organizing maps, including two surprising applications of mapping algorithms to standard computer science problems: combinatorial optimization and sorting. Contributors J. J. Atick, H. G. Barrow, H. U. Bauer, C. M. Bishop, H. J. Bray, J. Bruske, J. M. L. Budd, M. Budinich, V. Cherkassky, J. Cowan, R. Durbin, E. Erwin, G. J. Goodhill, T. Graepel, D. Grier, S. Kaski, T. Kohonen, H. Lappalainen, Z. Li, J. Lin, R. Linsker, S. P. Luttrell, D. J. C. MacKay, K. D. Miller, G. Mitchison, F. Mulier, K. Obermayer, C. Piepenbrock, H. Ritter, K. Schulten, T. J. Sejnowski, S. Smirnakis, G. Sommer, M. Svensen, R. Szeliski, A. Utsugi, C. K. I. Williams, L. Wiskott, L. Xu, A. Yuille, J. Zhang

Pattern Recognition by Self-Organizing Neural Networks presentsthe most recent advances in an area of research that is becoming vitally important in the fields ofcognitive science, neuroscience, artificial intelligence, and neural networks in general. The 19articles take up developments in competitive learning and computational maps, adaptive resonancetheory, and specialized architectures and biological connections. Introductorysurvey articles provide a framework for understanding the many models involved in various approachesto studying neural networks. These are followed in Part 2 by articles that form the foundation formodels of competitive learning and computational mapping, and recent articles by Kohonen, applyingthem to problems in speech recognition, and by Hecht-Nielsen, applying them to problems in designingadaptive lookup tables. Articles in Part 3 focus on adaptive resonance theory (ART) networks,selforganizing pattern recognition systems whose top-down template feedback signals guarantee theirstable learning in response to arbitrary sequences of input patterns. In Part 4, articles describeembedding ART modules into larger architectures and provide experimental evidence fromneurophysiology, event-related potentials, and psychology that support the prediction that ARTmechanisms exist in the brain. Contributors: J.-P. Banquet, G.A. Carpenter, S.Grossberg, R. Hecht-Nielsen, T. Kohonen, B. Kosko, T.W. Ryan, N.A. Schmajuk, W. Singer, D. Stork, C.von der Malsburg, C.L. Winter.