Human robot interaction through simple expressions for object recognition

Abstract

Abstract-Service robots need to be able to recognize and identify objects located within complex backgrounds. Since no single method may work in every situation, several methods need to be combined. However, there are several cases when autonomous recognition methods fail. We propose an interactive recognition method in these cases. To develop a natural Human Robot Interaction (HRI), it is necessary that the robot should unambiguously perceive the description of an object given by human. This paper reports on our experiment in which we examined the expressions humans use in describing ordinary objects. The results show that humans typically describe objects using one of multiple colors. The color is usually either that of the object background or that of the largest object portion. Based on these results, we describe our development of a robot vision system that can recognize objects when a user adopts simple expressions to describe the objects. This research suggests the importance of connecting 'symbolic expressions' with the 'real world' in human-robot interaction.

Object recognition in service robots: Conducting verbal interaction on color and spatial relationship

“…In order to be user-friendly, the robotic system should be able to understand such human expressions and locate objects without failure. For this, we have proposed a vision system that identifies multicolored objects even when the user mentions one color [4]. In this paper, we extend this research.…”

“…Based on this finding, we have developed a vision program to detect multicolored objects whose background or largest area color is the color specified by the user [4]. The background color is determined as the color of the region whose convex hull is the largest among neighboring regions in the color segmentation resultዊFig.…”

“…Yet, it remains difficult for vision systems to recognize objects in various natural conditions. In addressing this problem, we are currently working on an interactive object recognition system [3] [4]. Here the robot asks the user to verbally provide information about the object it cannot detect such as color and spatial relationship to other objects.…”

“…For example, we may want the robot to bring a bag of potato chips that has various colors. In order to test the assumption that users may not use complex expressions to describe such complex objects, we performed observation experiments on human-human interactions and found that humans usually use simple expressions to describe complex objects [4]. In particular, they usually used one major color to describe multi-colored objects.…”