In this paper, we describe experiments with methods for learning the appropriateness of behaviors based on a model of the current social situation. We first review different approaches for social robotics, and present a new approach based on situation modeling. We then review algorithms for social learning and propose three modifications to the classical Q-Learning algorithm. We describe five experiments with progressively complex algorithms for learning the appropriateness of behaviors. The first three experiments illustrate how social factors can be used to improve learning by controlling learning rate. In the fourth experiment we demonstrate that proper credit assignment improves the effectiveness of reinforcement learning for social interaction. In our fifth experiment we show that analogy can be used to accelerate learning rates in contexts composed of many situations.
Categories and Subject Descriptors
Situated Social Common SenseWith current technology, systems and services are unable to discriminate between appropriate and inappropriate behaviors. As a result, most attempts at proactive user services produce systems that are highly disruptive of human activity. In short, computing systems lack social common sense.Common sense is the collection of shared concepts and ideas that are accepted as correct by a community of people. Social common sense refers to the shared rules for polite, social interaction that implicitly rule behavior within a social group. To a large extent, such common sense is developed using implicit feedback during interaction between individuals. Our goal in this research is to develop methods to endow an artificial agent with the ability to acquire social common sense using the implicit feedback obtained from interaction with people. We believe that such methods can provide a foundation for socially polite man-machine interaction, and ultimately for other forms of cognitive abilities.In this paper, we propose to focus on a key aspect of social common sense: the ability to act appropriately in social situations. In this work, we have sought to train an association between behavior and social situation. Our approach for modeling social situations is inspired by the cognitive models for situation proposed by Johnson-Laird [14] in which situations are modeled as relations between entities. In previous work, we have generalized situation models with the introduction of the concept of "role" [8] and experimented with the use of machine learning techniques for automatically acquiring situation models. In this paper we extend this approach to the problem of learning social common sense. It is our intention that these methods may be used with any system that can spontaneously act to propose information or services.Rules for polite social interaction tend to be highly dependent on context, as well as specific to individuals or groups. Thus we have sought methods that would allow systems to learn the appropriateness of actions using the natural social feedback that people provide in mo...