Neural Networks In Optimization

People are facing more and more NP-complete or NP-hard problems of a combinatorial nature and of a continuous nature in economic, military and management practice. There are two ways in which one can enhance the efficiency of searching for the solutions of these problems. The first is to improve the speed and memory capacity of hardware. We all have witnessed the computer industry's amazing achievements with hardware and software developments over the last twenty years. On one hand many computers, bought only a few years ago, are being sent to elementary schools for children to learn the ABC's of computing. On the other hand, with economic, scientific and military developments, it seems that the increase of intricacy and the size of newly arising problems have no end. We all realize then that the second way, to design good algorithms, will definitely compensate for the hardware limitations in the case of complicated problems. It is the collective and parallel computation property of artificial neural net works that has activated the enthusiasm of researchers in the field of computer science and applied mathematics. It is hard to say that artificial neural networks are solvers of the above-mentioned dilemma, but at least they throw some new light on the difficulties we face. We not only anticipate that there will be neural computers with intelligence but we also believe that the research results of artificial neural networks might lead to new algorithms on von Neumann's computers.

A topical introduction on the ability of artificial neural networks to not only solve on-line a wide range of optimization problems but also to create new techniques and architectures. Provides in-depth coverage of mathematical modeling along with illustrative computer simulation results.

This book examines how nonlinear optimization techniques can be applied to training and testing neural networks. It includes both well-known and recently-developed network training methods including deterministic nonlinear optimization methods, stochastic nonlinear optimization methods, and advanced training schemes which combine both deterministic and stochastic components. The convergence analysis and convergence proofs of these techniques are presented as well as real applications of neural networks in areas such as pattern classification, bioinformatics, biomedicine, and finance. Nonlinear optimization methods are applied extensively in the design of training protocols for artificial neural networks used in industry and academia. Such techniques allow for the implementation of dynamic unsupervised neural network training without requiring the fine tuning of several heuristic parameters. "Nonlinear Optimization Approaches for Training Neural Networks" is a response to the growing demand for innovations in this area of research. This monograph presents a wide range of approaches to neural networks training providing theoretical justification for network behavior based on the theory of nonlinear optimization. It presents training algorithms, and theoretical results on their convergence and implementations through pseudocode. This approach offers the reader an explanation of the performance of the various methods, and a better understanding of the individual characteristics of the various methods, their differences/advantages and interrelationships. This improved perspective allows the reader to choose the best network training method without spending too much effort configuring highly sensitive heuristic parameters. This book can serve as an excellent guide for researchers, graduate students, and lecturers interested in the development of neural networks and their training.

This volume presents the set of final accepted papers for the tenth edition of the IWANN conference “International Work-Conference on Artificial neural Networks” held in Salamanca (Spain) during June 10–12, 2009. IWANN is a biennial conference focusing on the foundations, theory, models and applications of systems inspired by nature (mainly, neural networks, evolutionary and soft-computing systems). Since the first edition in Granada (LNCS 540, 1991), the conference has evolved and matured. The list of topics in the successive Call for - pers has also evolved, resulting in the following list for the present edition: 1. Mathematical and theoretical methods in computational intelligence. C- plex and social systems. Evolutionary and genetic algorithms. Fuzzy logic. Mathematics for neural networks. RBF structures. Self-organizing networks and methods. Support vector machines. 2. Neurocomputational formulations. Single-neuron modelling. Perceptual m- elling. System-level neural modelling. Spiking neurons. Models of biological learning. 3. Learning and adaptation. Adaptive systems. Imitation learning. Reconfig- able systems. Supervised, non-supervised, reinforcement and statistical al- rithms. 4. Emulation of cognitive functions. Decision making. Multi-agent systems. S- sor mesh. Natural language. Pattern recognition. Perceptual and motor functions (visual, auditory, tactile, virtual reality, etc.). Robotics. Planning motor control. 5. Bio-inspired systems and neuro-engineering. Embedded intelligent systems. Evolvable computing. Evolving hardware. Microelectronics for neural, fuzzy and bio-inspired systems. Neural prostheses. Retinomorphic systems. Bra- computer interfaces (BCI). Nanosystems. Nanocognitive systems.

This book discusses one of the major applications of artificial intelligence: the use of machine learning to extract useful information from multimodal data. It discusses the optimization methods that help minimize the error in developing patterns and classifications, which further helps improve prediction and decision-making. The book also presents formulations of real-world machine learning problems, and discusses AI solution methodologies as standalone or hybrid approaches. Lastly, it proposes novel metaheuristic methods to solve complex machine learning problems. Featuring valuable insights, the book helps readers explore new avenues leading toward multidisciplinary research discussions.

Optimization Techniques is a unique reference source to a diverse array of methods for achieving optimization, and includes both systems structures and computational methods. The text devotes broad coverage toa unified view of optimal learning, orthogonal transformation techniques, sequential constructive techniques, fast back propagation algorithms, techniques for neural networks with nonstationary or dynamic outputs, applications to constraint satisfaction,optimization issues and techniques for unsupervised learning neural networks, optimum Cerebellar Model of Articulation Controller systems, a new statistical theory of optimum neural learning, and the role of the Radial Basis Function in nonlinear dynamical systems.This volume is useful for practitioners, researchers, and students in industrial, manufacturing, mechanical, electrical, and computer engineering. Provides in-depth treatment of theoretical contributions to optimal learning for neural network systems Offers a comprehensive treatment of orthogonal transformation techniques for the optimization of neural network systems Includes illustrative examples and comprehensive treatment of sequential constructive techniques for optimization of neural network systems Presents a uniquely comprehensive treatment of the highly effective fast back propagation algorithms for the optimization of neural network systems Treats, in detail, optimization techniques for neural network systems with nonstationary or dynamic inputs Covers optimization techniques and applications of neural network systems in constraint satisfaction

This book provides an in-depth analysis of the current evolutionary machine learning techniques. Discussing the most highly regarded methods for classification, clustering, regression, and prediction, it includes techniques such as support vector machines, extreme learning machines, evolutionary feature selection, artificial neural networks including feed-forward neural networks, multi-layer perceptron, probabilistic neural networks, self-optimizing neural networks, radial basis function networks, recurrent neural networks, spiking neural networks, neuro-fuzzy networks, modular neural networks, physical neural networks, and deep neural networks. The book provides essential definitions, literature reviews, and the training algorithms for machine learning using classical and modern nature-inspired techniques. It also investigates the pros and cons of classical training algorithms. It features a range of proven and recent nature-inspired algorithms used to train different types of artificial neural networks, including genetic algorithm, ant colony optimization, particle swarm optimization, grey wolf optimizer, whale optimization algorithm, ant lion optimizer, moth flame algorithm, dragonfly algorithm, salp swarm algorithm, multi-verse optimizer, and sine cosine algorithm. The book also covers applications of the improved artificial neural networks to solve classification, clustering, prediction and regression problems in diverse fields.

This book delivers the state of the art in deep learning (DL) methods hybridized with evolutionary computation (EC). Over the last decade, DL has dramatically reformed many domains: computer vision, speech recognition, healthcare, and automatic game playing, to mention only a few. All DL models, using different architectures and algorithms, utilize multiple processing layers for extracting a hierarchy of abstractions of data. Their remarkable successes notwithstanding, these powerful models are facing many challenges, and this book presents the collaborative efforts by researchers in EC to solve some of the problems in DL. EC comprises optimization techniques that are useful when problems are complex or poorly understood, or insufficient information about the problem domain is available. This family of algorithms has proven effective in solving problems with challenging characteristics such as non-convexity, non-linearity, noise, and irregularity, which dampen the performance of most classic optimization schemes. Furthermore, EC has been extensively and successfully applied in artificial neural network (ANN) research —from parameter estimation to structure optimization. Consequently, EC researchers are enthusiastic about applying their arsenal for the design and optimization of deep neural networks (DNN). This book brings together the recent progress in DL research where the focus is particularly on three sub-domains that integrate EC with DL: (1) EC for hyper-parameter optimization in DNN; (2) EC for DNN architecture design; and (3) Deep neuroevolution. The book also presents interesting applications of DL with EC in real-world problems, e.g., malware classification and object detection. Additionally, it covers recent applications of EC in DL, e.g. generative adversarial networks (GAN) training and adversarial attacks. The book aims to prompt and facilitate the research in DL with EC both in theory and in practice.