Recent Advances In Learning And Control

This volume is composed of invited papers on learning and control. The contents form the proceedings of a workshop held in January 2008, in Hyderabad that honored the 60th birthday of Doctor Mathukumalli Vidyasagar. The 14 papers, written by international specialists in the field, cover a variety of interests within the broader field of learning and control. The diversity of the research provides a comprehensive overview of a field of great interest to control and system theorists.

This volume is composed of invited papers on learning and control. The contents form the proceedings of a workshop held in January 2008, in Hyderabad that honored the 60th birthday of Doctor Mathukumalli Vidyasagar. The 14 papers, written by international specialists in the field, cover a variety of interests within the broader field of learning and control. The diversity of the research provides a comprehensive overview of a field of great interest to control and system theorists.

This book collects recent theoretical advances and concrete applications of learning automata (LAs) in various areas of computer science, presenting a broad treatment of the computer science field in a survey style. Learning automata (LAs) have proven to be effective decision-making agents, especially within unknown stochastic environments. The book starts with a brief explanation of LAs and their baseline variations. It subsequently introduces readers to a number of recently developed, complex structures used to supplement LAs, and describes their steady-state behaviors. These complex structures have been developed because, by design, LAs are simple units used to perform simple tasks; their full potential can only be tapped when several interconnected LAs cooperate to produce a group synergy. In turn, the next part of the book highlights a range of LA-based applications in diverse computer science domains, from wireless sensor networks, to peer-to-peer networks, to complex social networks, and finally to Petri nets. The book accompanies the reader on a comprehensive journey, starting from basic concepts, continuing to recent theoretical findings, and ending in the applications of LAs in problems from numerous research domains. As such, the book offers a valuable resource for all computer engineers, scientists, and students, especially those whose work involves the reinforcement learning and artificial intelligence domains.

Recent Advances in Robot Learning contains seven papers on robot learning written by leading researchers in the field. As the selection of papers illustrates, the field of robot learning is both active and diverse. A variety of machine learning methods, ranging from inductive logic programming to reinforcement learning, is being applied to many subproblems in robot perception and control, often with objectives as diverse as parameter calibration and concept formulation. While no unified robot learning framework has yet emerged to cover the variety of problems and approaches described in these papers and other publications, a clear set of shared issues underlies many robot learning problems. Machine learning, when applied to robotics, is situated: it is embedded into a real-world system that tightly integrates perception, decision making and execution. Since robot learning involves decision making, there is an inherent active learning issue. Robotic domains are usually complex, yet the expense of using actual robotic hardware often prohibits the collection of large amounts of training data. Most robotic systems are real-time systems. Decisions must be made within critical or practical time constraints. These characteristics present challenges and constraints to the learning system. Since these characteristics are shared by other important real-world application domains, robotics is a highly attractive area for research on machine learning. On the other hand, machine learning is also highly attractive to robotics. There is a great variety of open problems in robotics that defy a static, hand-coded solution. Recent Advances in Robot Learning is an edited volume of peer-reviewed original research comprising seven invited contributions by leading researchers. This research work has also been published as a special issue of Machine Learning (Volume 23, Numbers 2 and 3).

This volume is composed of invited papers on learning and control. The contents form the proceedings of a workshop held in January 2008, in Hyderabad that honored the 60th birthday of Doctor Mathukumalli Vidyasagar. The 14 papers, written by international specialists in the field, cover a variety of interests within the broader field of learning and control. The diversity of the research provides a comprehensive overview of a field of great interest to control and system theorists.

The role of manufacturing in a country’s economy and societal development has long been established through their wealth generating capabilities. To enhance and widen our knowledge of materials and to increase innovation and responsiveness to ever-increasing international needs, more in-depth studies of functionally graded materials/tailor-made materials, recent advancements in manufacturing processes and new design philosophies are needed at present. The objective of this volume is to bring together experts from academic institutions, industries and research organizations and professional engineers for sharing of knowledge, expertise and experience in the emerging trends related to design, advanced materials processing and characterization, and advanced manufacturing processes.

Inthesummerof2008,reinforcementlearningresearchersfromaroundtheworld gathered in the north of France for a week of talks and discussions on reinfor- ment learning, on how it could be made more e?cient, applied to a broader range of applications, and utilized at more abstract and symbolic levels. As a participant in this 8th European Workshop on Reinforcement Learning, I was struck by both the quality and quantity of the presentations. There were four full days of short talks, over 50 in all, far more than there have been at any p- vious meeting on reinforcement learning in Europe, or indeed, anywhere else in the world. There was an air of excitement as substantial progress was reported in many areas including Computer Go, robotics, and ?tted methods. Overall, the work reported seemed to me to be an excellent, broad, and representative sample of cutting-edge reinforcement learning research. Some of the best of it is collected and published in this volume. The workshopandthe paperscollectedhere provideevidence thatthe ?eldof reinforcement learning remains vigorous and varied. It is appropriate to re?ect on some of the reasons for this. One is that the ?eld remains focused on a pr- lem — sequential decision making — without prejudice as to solution methods. Another is the existence of a common terminology and body of theory.

0. 1 Introduction Although the general optimal solution of the ?ltering problem for nonlinear state and observation equations confused with white Gaussian noises is given by the Kushner equation for the conditional density of an unobserved state with respect to obser- tions (see [48] or [41], Theorem 6. 5, formula (6. 79) or [70], Subsection 5. 10. 5, formula (5. 10. 23)), there are a very few known examples of nonlinear systems where the Ku- ner equation can be reduced to a ?nite-dimensional closed system of ?ltering eq- tions for a certain number of lower conditional moments. The most famous result, the Kalman-Bucy ?lter [42], is related to the case of linear state and observation equations, where only two moments, the estimate itself and its variance, form a closed system of ?ltering equations. However, the optimal nonlinear ?nite-dimensional ?lter can be - tained in some other cases, if, for example, the state vector can take only a ?nite number of admissible states [91] or if the observation equation is linear and the drift term in the 2 2 state equation satis?es the Riccati equation df /dx + f = x (see [15]). The complete classi?cation of the “general situation” cases (this means that there are no special - sumptions on the structure of state and observation equations and the initial conditions), where the optimal nonlinear ?nite-dimensional ?lter exists, is given in [95].