Intrinsically Motivated Reinforcement Learning A Promising Framework For Developmental Robot Learning

It has become clear to researchers in robotics and adaptive behaviour that current approaches are yielding systems with limited autonomy and capacity for self-improvement. To learn autonomously and in a cumulative fashion is one of the hallmarks of intelligence, and we know that higher mammals engage in exploratory activities that are not directed to pursue goals of immediate relevance for survival and reproduction but are instead driven by intrinsic motivations such as curiosity, interest in novel stimuli or surprising events, and interest in learning new behaviours. The adaptive value of such intrinsically motivated activities lies in the fact that they allow the cumulative acquisition of knowledge and skills that can be used later to accomplish fitness-enhancing goals. Intrinsic motivations continue during adulthood, and in humans they underlie lifelong learning, artistic creativity, and scientific discovery, while they are also the basis for processes that strongly affect human well-being, such as the sense of competence, self-determination, and self-esteem. This book has two aims: to present the state of the art in research on intrinsically motivated learning, and to identify the related scientific and technological open challenges and most promising research directions. The book introduces the concept of intrinsic motivation in artificial systems, reviews the relevant literature, offers insights from the neural and behavioural sciences, and presents novel tools for research. The book is organized into six parts: the chapters in Part I give general overviews on the concept of intrinsic motivations, their function, and possible mechanisms for implementing them; Parts II, III, and IV focus on three classes of intrinsic motivation mechanisms, those based on predictors, on novelty, and on competence; Part V discusses mechanisms that are complementary to intrinsic motivations; and Part VI introduces tools and experimental frameworks for investigating intrinsic motivations. The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots. The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots.

The aim of this Research Topic for Frontiers in Psychology under the section of Cognitive Science and Frontiers in Neurorobotics is to present state-of-the-art research, whether theoretical, empirical, or computational investigations, on open-ended development driven by intrinsic motivations. The topic will address questions such as: How do motivations drive learning? How are complex skills built up from a foundation of simpler competencies? What are the neural and computational bases for intrinsically motivated learning? What is the contribution of intrinsic motivations to wider cognition? Autonomous development and lifelong open-ended learning are hallmarks of intelligence. Higher mammals, and especially humans, engage in activities that do not appear to directly serve the goals of survival, reproduction, or material advantage. Rather, a large part of their activity is intrinsically motivated - behavior driven by curiosity, play, interest in novel stimuli and surprising events, autonomous goal-setting, and the pleasure of acquiring new competencies. This allows the cumulative acquisition of knowledge and skills that can later be used to accomplish fitness-enhancing goals. Intrinsic motivations continue during adulthood, and in humans artistic creativity, scientific discovery, and subjective well-being owe much to them. The study of intrinsically motivated behavior has a long history in psychological and ethological research, which is now being reinvigorated by perspectives from neuroscience, artificial intelligence and computer science. For example, recent neuroscientific research is discovering how neuromodulators like dopamine and noradrenaline relate not only to extrinsic rewards but also to novel and surprising events, how brain areas such as the superior colliculus and the hippocampus are involved in the perception and processing of events, novel stimuli, and novel associations of stimuli, and how violations of predictions and expectations influence learning and motivation. Computational approaches are characterizing the space of possible reinforcement learning algorithms and their augmentation by intrinsic reinforcements of different kinds. Research in robotics and machine learning is yielding systems with increasing autonomy and capacity for self-improvement: artificial systems with motivations that are similar to those of real organisms and support prolonged autonomous learning. Computational research on intrinsic motivation is being complemented by, and closely interacting with, research that aims to build hierarchical architectures capable of acquiring, storing, and exploiting the knowledge and skills acquired through intrinsically motivated learning. Now is an important moment in the study of intrinsically motivated open-ended development, requiring contributions and integration across a large number of fields within the cognitive sciences. This Research Topic aims to contribute to this effort by welcoming papers carried out with ethological, psychological, neuroscientific and computational approaches, as well as research that cuts across disciplines and approaches.

Even since computers were invented, many researchers have been trying to understand how human beings learn and many interesting paradigms and approaches towards emulating human learning abilities have been proposed. The ability of learning is one of the central features of human intelligence, which makes it an important ingredient in both traditional Artificial Intelligence (AI) and emerging Cognitive Science. Machine Learning (ML) draws upon ideas from a diverse set of disciplines, including AI, Probability and Statistics, Computational Complexity, Information Theory, Psychology and Neurobiology, Control Theory and Philosophy. ML involves broad topics including Fuzzy Logic, Neural Networks (NNs), Evolutionary Algorithms (EAs), Probability and Statistics, Decision Trees, etc. Real-world applications of ML are widespread such as Pattern Recognition, Data Mining, Gaming, Bio-science, Telecommunications, Control and Robotics applications. This books reports the latest developments and futuristic trends in ML.

This book constitutes the refereed proceedings of the 9th European Conference on Artificial Life, ECAL 2007, held in Lisbon, Portugal. The 125 revised full papers cover morphogenesis and development, robotics and autonomous agents, evolutionary computation and theory, cellular automata, models of biological systems and their applications, ant colony and swarm systems, evolution of communication, simulation of social interactions, self-replication, artificial chemistry.

This book constitutes the refereed proceedings of the 9th International Conference on Simulation of Adaptive Behavior, SAB 2006. The 35 revised full papers and 35 revised poster papers presented are organized in topical sections on the animat approach to adaptive behaviour, perception and motor control, action selection and behavioral sequences, navigation and internal world models, learning and adaptation, evolution, collective and social behaviours, applied adaptive behavior and more.

This book constitutes the proceedings of the 21st International Conference on Practical Applications of Agents and Multi-Agent Systems, PAAMS 2023, held in Guimaraes, Portugal, in July 2023. The 32 full papers in this book were reviewed and selected from 70 submissions. 5 demonstration papers are also included in this volume. The papers deal with the application and validation of agent-based models, methods, and technologies in a number of key applications areas, including: advanced models and learning, agent-based programming, decision-making, education and social interactions, formal and theoretic models, health and safety, mobility and the city, swarms and task allocation.

This book constitutes the proceedings of the International Conference on Adaptive and Intelligent Systems, ICAIS 2011, held in Klagenfurt, Austria, in September 2011. The 36 full papers included in these proceedings together with the abstracts of 4 invited talks, were carefully reviewed and selected from 72 submissions. The contributions are organized under the following topical sections: incremental learning; adaptive system architecture; intelligent system engineering; data mining and pattern recognition; intelligent agents; and computational intelligence.