Learning Classifier Systems

This volume brings together recent theoretical work in Learning Classifier Systems (LCS), which is a Machine Learning technique combining Genetic Algorithms and Reinforcement Learning. It includes self-contained background chapters on related fields (reinforcement learning and evolutionary computation) tailored for a classifier systems audience and written by acknowledged authorities in their area - as well as a relevant historical original work by John Holland.

Learning Classifier Systems (LCS) are a machine learning paradigm introduced by John Holland in 1976. They are rule-based systems in which learning is viewed as a process of ongoing adaptation to a partially unknown environment through genetic algorithms and temporal difference learning. This book provides a unique survey of the current state of the art of LCS and highlights some of the most promising research directions. The first part presents various views of leading people on what learning classifier systems are. The second part is devoted to advanced topics of current interest, including alternative representations, methods for evaluating rule utility, and extensions to existing classifier system models. The final part is dedicated to promising applications in areas like data mining, medical data analysis, economic trading agents, aircraft maneuvering, and autonomous robotics. An appendix comprising 467 entries provides a comprehensive LCS bibliography.

This accessible introduction shows the reader how to understand, implement, adapt, and apply Learning Classifier Systems (LCSs) to interesting and difficult problems. The text builds an understanding from basic ideas and concepts. The authors first explore learning through environment interaction, and then walk through the components of LCS that form this rule-based evolutionary algorithm. The applicability and adaptability of these methods is highlighted by providing descriptions of common methodological alternatives for different components that are suited to different types of problems from data mining to autonomous robotics. The authors have also paired exercises and a simple educational LCS (eLCS) algorithm (implemented in Python) with this book. It is suitable for courses or self-study by advanced undergraduate and postgraduate students in subjects such as Computer Science, Engineering, Bioinformatics, and Cybernetics, and by researchers, data analysts, and machine learning practitioners.

Anticipatory Learning Classifier Systems describes the state of the art of anticipatory learning classifier systems-adaptive rule learning systems that autonomously build anticipatory environmental models. An anticipatory model specifies all possible action-effects in an environment with respect to given situations. It can be used to simulate anticipatory adaptive behavior. Anticipatory Learning Classifier Systems highlights how anticipations influence cognitive systems and illustrates the use of anticipations for (1) faster reactivity, (2) adaptive behavior beyond reinforcement learning, (3) attentional mechanisms, (4) simulation of other agents and (5) the implementation of a motivational module. The book focuses on a particular evolutionary model learning mechanism, a combination of a directed specializing mechanism and a genetic generalizing mechanism. Experiments show that anticipatory adaptive behavior can be simulated by exploiting the evolving anticipatory model for even faster model learning, planning applications, and adaptive behavior beyond reinforcement learning. Anticipatory Learning Classifier Systems gives a detailed algorithmic description as well as a program documentation of a C++ implementation of the system.

Learning classi er systems are rule-based systems that exploit evolutionary c- putation and reinforcement learning to solve di cult problems. They were - troduced in 1978 by John H. Holland, the father of genetic algorithms, and since then they have been applied to domains as diverse as autonomous robotics, trading agents, and data mining. At the Second International Workshop on Learning Classi er Systems (IWLCS 99), held July 13, 1999, in Orlando, Florida, active researchers reported on the then current state of learning classi er system research and highlighted some of the most promising research directions. The most interesting contri- tions to the meeting are included in the book Learning Classi er Systems: From Foundations to Applications, published as LNAI 1813 by Springer-Verlag. The following year, the Third International Workshop on Learning Classi er Systems (IWLCS 2000), held September 15{16 in Paris, gave participants the opportunity to discuss further advances in learning classi er systems. We have included in this volume revised and extended versions of thirteen of the papers presented at the workshop.

The field called Learning Classifier Systems is populated with romantics. Why shouldn't it be possible for computer programs to adapt, learn, and develop while interacting with their environments? In particular, why not systems that, like organic populations, contain competing, perhaps cooperating, entities evolving together? John Holland was one of the earliest scientists with this vision, at a time when so-called artificial intelligence was in its infancy and mainly concerned with preprogrammed systems that didn't learn. that, like organisms, had sensors, took Instead, Holland envisaged systems actions, and had rich self-generated internal structure and processing. In so doing he foresaw and his work prefigured such present day domains as reinforcement learning and embedded agents that are now displacing the older "standard Af' . One focus was what Holland called "classifier systems": sets of competing rule like "classifiers", each a hypothesis as to how best to react to some aspect of the environment--or to another rule. The system embracing such a rule "popu lation" would explore its available actions and responses, rewarding and rating the active rules accordingly. Then "good" classifiers would be selected and re produced, mutated and even crossed, a la Darwin and genetics, steadily and reliably increasing the system's ability to cope.

This book constitutes the thoroughly refereed joint post-conference proceedings of two consecutive International Workshops on Learning Classifier Systems that took place in Seattle, WA, USA in July 2006, and in London, UK, in July 2007 - all hosted by the Genetic and Evolutionary Computation Conference, GECCO. The 14 revised full papers presented were carefully reviewed and selected from the workshop contributions. The papers are organized in topical sections on knowledge representation, analysis of the system, mechanisms, new directions, as well as applications.

Rule-basedevolutionaryonlinelearningsystems,oftenreferredtoasMichig- style learning classi?er systems (LCSs), were proposed nearly thirty years ago (Holland, 1976; Holland, 1977) originally calling them cognitive systems. LCSs combine the strength of reinforcement learning with the generali- tion capabilities of genetic algorithms promising a ?exible, online general- ing, solely reinforcement dependent learning system. However, despite several initial successful applications of LCSs and their interesting relations with a- mal learning and cognition, understanding of the systems remained somewhat obscured. Questions concerning learning complexity or convergence remained unanswered. Performance in di?erent problem types, problem structures, c- ceptspaces,andhypothesisspacesstayednearlyunpredictable. Thisbookhas the following three major objectives: (1) to establish a facetwise theory - proachforLCSsthatpromotessystemanalysis,understanding,anddesign;(2) to analyze, evaluate, and enhance the XCS classi?er system (Wilson, 1995) by the means of the facetwise approach establishing a fundamental XCS learning theory; (3) to identify both the major advantages of an LCS-based learning approach as well as the most promising potential application areas. Achieving these three objectives leads to a rigorous understanding of LCS functioning that enables the successful application of LCSs to diverse problem types and problem domains. The quantitative analysis of XCS shows that the inter- tive, evolutionary-based online learning mechanism works machine learning competitively yielding a low-order polynomial learning complexity. Moreover, the facetwise analysis approach facilitates the successful design of more - vanced LCSs including Holland’s originally envisioned cognitive systems. Martin V.