No abstract

We propose a new method to categorize continuous numeric percepts for Q-learning, where percept vectors are classified into categories on the basis of fuzzy ART and Q-learning uses categories as states to acquire rules for agent behavior. For efficient learning, we modify fuzzy ART to reduce the number of categories without deteriorating the efficiency of reinforcement learning. In our modification, a vigilance parameter is defined for each category in order to control the size of a category and it is updated during learning. The method to update a vigilance parameter is based on category integration, which contributes to reducing the number of categories. Here, we define the similarity for any category pair to judge whether category integration should be performed or not. When two categories are integrated into a new category, a vigilance parameter for the category is calculated and categories used for integration are discarded, so that the number of categories is reduced without restricting the number of categories. Experimental results show that Q-learning with modified fuzzy ART acquires good rules for agent behavior more efficiently than Q-learning with ordinary fuzzy ART, although the number of categories generated by modified fuzzy ART is much less than that generated by ordinary one.

No abstract

We propose a new method SSED (State Segmentation based on Euclidean Distance) to categorize continuous numeric percepts for Q-learning, where percept vectors are classified into categories and Q-learning uses categories as states to acquire rules for agent behavior. In SSED, categories are represented by hyper-spheres. A percept vector is classified into a category that covers the vector and is the nearest to it. For efficient reinforcement learning, category merging is provided with SSED, where the number of parameters to control category merging in SSED is fewer than that in fuzzy ART with category merging. In addition, match tracking is incorporated into SSED in order to specialize a category. SSED is combined with Q-learning and it is compared with some state segmentation methods. Experimental results show that Q-learning with SSED learns good rules for agent behavior more efficiently than other methods.

No abstract