Comparison of two methods for computing action values in XCS with code-fragment actions

Iqbal, Muhammad; Browne, Will N.; Zhang, Mengjie

doi:10.1145/2464576.2482702

Cited by 6 publications

(12 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previously, we implemented this scheme to encode the action in a classifier rule, which produced optimal populations in discrete domain problems [52], [61], [62] as well as in continuous domain problems [63], but this did not lead to simple scaling. In our previous code-fragment conditions work [60], [64], we used a separate population of code fragments, which limited the number of available code fragments, resulting in a system that was not able to learn the complex, large-scale problems.…”

Section: ) Previous Work On Code-fragment Based Xcsmentioning

confidence: 99%

Reusing Building Blocks of Extracted Knowledge to Solve Complex, Large-Scale Boolean Problems

Iqbal

Browne

Zhang

2014

IEEE Trans. Evol. Computat.

Self Cite

137

147

View full text Add to dashboard Cite

Evolutionary computation techniques have had limited capabilities in solving large-scale problems due to the large search space demanding large memory and much longer training times. In the work presented here, a genetic programming like rich encoding scheme has been constructed to identify building blocks of knowledge in a learning classifier system. The fitter building blocks from the learning system trained against smaller problems have been utilized in a higher complexity problem in the domain to achieve scalable learning. The proposed system has been examined and evaluated on four different Boolean problem domains: 1) multiplexer, 2) majority-on, 3) carry, and 4) evenparity problems. The major contribution of this paper is to successfully extract useful building blocks from smaller problems and reuse them to learn more complex large-scale problems in the domain, e.g., 135-bit multiplexer problem, where the number of possible instances is 2 135 ≈ 4 × 10 40 , is solved by reusing the extracted knowledge from the learned lower level solutions in the domain. Autonomous scaling is, for the first time, shown to be possible in learning classifier systems. It improves effectiveness and reduces the number of training instances required in large problems, but requires more time due to its sequential build-up of knowledge.

show abstract

Section: ) Previous Work On Code-fragment Based Xcsmentioning

confidence: 99%

Reusing Building Blocks of Extracted Knowledge to Solve Complex, Large-Scale Boolean Problems

Iqbal

Browne

Zhang

2014

IEEE Trans. Evol. Computat.

Self Cite

137

147

View full text Add to dashboard Cite

show abstract

“…The action value of the classifier was determined by evaluating the action code tree [9]. In order to achieve this, it was necessary to replace the terminal symbols with corresponding binary bits from the associated condition in the classifier rule [5], [7].…”

Section: Introductionmentioning

confidence: 99%

Reusing learned functionality in XCS

Alvarez

Browne

Zhang

2014

Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation

Self Cite

View full text Add to dashboard Cite

This paper expands on work previously conducted on the XCS system using code fragments, which are GP-like trees that encapsulate building blocks of knowledge. The usage of code fragments in the XCS system enabled the solution of previously intractable, complex, boolean problems, e.g. the 135 bit multiplexer domain. However, it was not previously possible to replace functionality at nodes with learned relationships, which restricted scaling to larger problems and related domains. The aim of this paper is to reuse learned rule sets as functions. The functions are to be stored along with the code fragments produced as a solution for a problem. The results show for the first time that these learned functions can be reused in the inner nodes of the code fragment trees. The results are encouraging as there was no statistically significant difference in terms of classification. For the simpler problems the new system XCSCF 2 , required much less instances than the XCSCFC to solve the problems. However, for the more complex problems, the XCSCF 2 required more instances than XCSCFC; but the additional time was not prohibitive for the continued development of this approach. The main contribution of this investigation is that functions can be learned and later reused in the inner nodes of a code fragment tree. This is anticipated to lead to a reduced search space and increased performance both in terms of instances needed to solve a problem and classification accuracy.

show abstract

“…The system provided more expressivity for the action part while providing an unanticipated benefit. In certain XCSCFA configurations the final population of classifiers was autonomously divided into optimal and sub-optimal subpopulations [45]. This eased the process of simplification, see Chapter 2.…”

Section: Scopementioning

confidence: 99%

“…The action value of the classifier was determined by evaluating the action CF [47]. In order to achieve this, it was necessary to populate the CFA's terminal symbols [42], [45]. The terminals in the CF tree could be replaced with either the corresponding bits from the environment message or with bits from the classifier condition.…”

Section: Code Fragment Based Systemsmentioning

confidence: 99%

“…Another advantage of utilizing CFs in the action of a classifier is the autonomous separation of optimal and sub-optimal classifiers in the final population, which is due to the consistency of action formed by these classifiers [45]. This eventually results in the optimum rule set of the maximally general, compact and accurate classifiers, see Table 2.1.…”

Section: Code Fragment Based Systemsmentioning

confidence: 99%

See 1 more Smart Citation

Developing the Capability to Scale and Reuse Learned Knowledge and Functionality in Learning Classifier Systems

Alvarez¹

View full text Add to dashboard Cite

<p>Learning is an important activity through which humanity has incrementally improved accomplishing tasks by adapting knowledge and methods based on the related feedback. Although learning is natural to humans, it is much more difficult to achieve in the technological world as tasks are often learned in isolation. Software is capable of learning novel techniques and algorithms in order to solve these basic, individual problems, however transferring said knowledge to other problems in the same or related domains presents challenges. Solutions often cannot be enumerated to discover the best one as many problems of interest can be intractable in terms of the resources needed to successfully complete them. However, many such problems contain key building blocks of knowledge that can be leveraged to achieve a suitable solution. These building blocks encapsulate important structural regularities of the problem. A technique that can learn these regularities without enumeration,may produce general solutions that apply to similar problems of any length. This implies reusing learned information. In order to reuse learned blocks of knowledge, it is important that a program be scalable and flexible. This requires a program capable of taking knowledge from a previous task and applying it to a more complex problem or a problem with a similar pattern. This is anticipated to enable the program to complete the new task in a practical amount of time and with reasonable amounts of resources. In machine learning, the degree of human intervention in solving problems is often important in many tasks. It is generally necessary for a human to provide input to direct and improve learning. In the field of Developmental Learning there is the idea known as the Threshold Concept (TC). A TC is transformative information which advocates learning. TCs are important because without them, the learner cannot progress. In addition, TCs need to be learned in a particular order, much like a curriculum, thus providing the student with viable progress towards learning more difficult ideas at a faster pace than otherwise. Therefore, human input to a learning algorithm can be to partition a problem into constituent subproblems. This is a principal concept of Layered Learning (LL),where a sequence of sub-problems are learned. The sub-problems are self-contained stages which have been separated by a human. This technique is necessary for tasks in which learning a direct mapping from inputs to outputs is intractable given existing learning algorithms. One of the first artificial learning systems developed is Learning Classifier Systems (LCSs). Past work has extended LCSs to provide more expressivity by using richer representations. One such representation is tree-based and is common to the Genetic Programming (GP) technique. GP is part of the Evolutionary Computation (EC) paradigm and produces solutions represented by trees. The tree nodes can contain functions, and leaf nodes problem features, giving GP a rich representation. A more recent technique is Code Fragments (CFs). CFs are GP-like sub-trees with an initial maximum height of two. Initially, CFs contained hard-coded functions at the root nodes and problem features or previously learned CFs at the leaf nodes of the sub-trees. CFs provided improved expressivity and scalability over the original ternary alphabet used by LCSs. Additionally, CF-based systems have successfully learned previously intractable problems, e.g. 135-bit multiplexer. Although CFs have provided increased scalability, they suffer from a structural weakness. As the problem scales, the chains of CFs grow to intractable lengths. This means that at some point the LCS will stop learning. In addition, CFs were originally meant to scale to more complex problems in the same domain. However, it is advantageous to compile cross-domain solutions, as the regularities of a problem might be from different domains to that expressed by the data. The proposed thesis is that a CF-based LCS can scale to complex problems by reusing learned solutions of problems as functions at the inner nodes of CFs together with compaction and Layered Learning. The overall goal is divided into the following three sub-goals: reuse learned functionality from smaller problems in the root nodes of CF sub-trees, identify a compaction technique that facilitates reduced solution size for improved evaluation time of CFs and develop a layered learning methodology for a CF system, which will be demonstrated by learning a general solution to an intractable problem, i.e. n-bit Multiplexer. In this novel work, Code Fragments are extended to include learned functionality at the root nodes of the sub-trees in a technique known as XCSCF². A new compaction technique is designed, which produces an equivalent set of ternary rules from CF rules. This technique is known as XCSCF3. The work culminates with a new technique XCSCF*, which combines Layered Learning, Code Fragments and Transfer Learning (TL) of knowledge and functionality to produce scalable and general solutions, i.e. to the n-bit multiplexer problem. The novel ideas are tested with the multiplexer and hidden multiplexer problems. These problems are chosen because they are difficult due to epistasis, sparsity and non-linearity. Therefore they provide ample opportunity for testing the new contributions. The thesis work has shown that CFs can be used in various ways to increase scalability and to discover solutions to complex problems. Specifically the following three contributions were produced: learned functionality was captured in LCS populations from smaller problems and was reused in the root nodes of CF sub-trees. An online compaction technique that facilitates reduced evaluation time of CFs was designed. A layered learning method to train a CF system in a manner leading to a general solution was developed. This was demonstrated through learning a solution to a previously intractable problem, i.e. the n-bit Multiplexer. The thesis concludes with suggestions for future work aimed at providing better scalability when using compaction techniques.</p>

show abstract

Comparison of two methods for computing action values in XCS with code-fragment actions

Cited by 6 publications

References 21 publications

Reusing Building Blocks of Extracted Knowledge to Solve Complex, Large-Scale Boolean Problems

Reusing Building Blocks of Extracted Knowledge to Solve Complex, Large-Scale Boolean Problems

Reusing learned functionality in XCS

Developing the Capability to Scale and Reuse Learned Knowledge and Functionality in Learning Classifier Systems

Contact Info

Product

Resources

About