Vision-speech system adapting to the user and environment for service robots

Yoshizaki, Mitsutoshi; Nakamura, Akio; Kuno, Yoshinori

doi:10.1109/iros.2003.1248823

Cited by 13 publications

(16 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among such utterances, we deal with ellipsis and deixis that may appear in the speech interface for the helper robot [6].…”

Section: Inexplicit Utterancesmentioning

confidence: 99%

“…Gibson has classified things to be perceived into five categories: places, attached objects, detached objects, persisting substances, and events [7]. We use Gibson's terminology in this paper as in our previous research [6]. The objects in the former group are detached objects, and those in the latter are attached objects.…”

Section: Information Obtained By Visionmentioning

confidence: 99%

“…We are developing a helper robot that brings the object that the user asks through speech [6]. In this paper, we present a speech interface for the robot that allows the user to use utterances with such vision-derived inexplicitness.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Understanding inexplicit utterances using vision for helper robots

Hanafiah¹,

Yamazaki²,

Nakamura³

et al. 2004

Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004.

Self Cite

View full text Add to dashboard Cite

Speech interfaces should have a capability of dealing with inexplicit utterances including such as ellipsis and deixis since they are common phenomena in our daily conversation. Their resolution using context and a priori knowledge has been investigated in the fields of natural language and speech understanding. However, there are utterances that cannot be understood by such symbol processing alone. In this paper, we consider inexplicit utterances caused from the fact that humans have vision. If we are certain that the listeners share some visual information, we often omit or mention ambiguously things about it in our utterances. We propose a method of understanding speech with such ambiguities using computer vision. It tracks the human's gaze direction, detecting objects in the direction. It also recognizes the human's actions.Based on these bits of visual information, it understands the human's inexplicit utterances. Experimental results show that the method helps to realize human-friendly speech interfaces.

show abstract

“…Among such utterances, we deal with ellipsis and deixis that may appear in the speech interface for the helper robot [6].…”

Section: Inexplicit Utterancesmentioning

confidence: 99%

Section: Information Obtained By Visionmentioning

confidence: 99%

See 1 more Smart Citation

Understanding inexplicit utterances using vision for helper robots

Hanafiah¹,

Yamazaki²,

Nakamura³

et al. 2004

Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Multimodal interfaces [1][12] [13] are considered strong candidates. Thus, we have been developing a helper robot that carries out tasks ordered by the user through voice and/or gestures [9][15] [18] [19]. In addition to gesture recognition, such robots need to have vision systems that can recognize the objects mentioned in speech.…”

Section: Introductionmentioning

confidence: 99%

“…It is, however, difficult to realize vision systems that can work in various conditions. Thus, we have proposed to use the human user's assistance through speech [9][15] [18] [19]. When the vision system cannot achieve a task, the robot makes a question to the user so that the natural response by the user can give helpful information for its vision system.…”

Section: Introductionmentioning

confidence: 99%

Interactive vision to detect target objects for helper robots

Hossain

Kurnia

Nakamura

et al. 2005

Proceedings of the 7th International Conference on Multimodal Interfaces

View full text Add to dashboard Cite

An effective human-robot interaction is essential for wide penetration of service robots into the market. Such robots need vision systems to recognize objects. It is, however, difficult to realize vision systems that can work in various conditions. More robust techniques of object recognition and image segmentation are essential. Thus, we have proposed to use the human user's assistance for object recognition through speech. The robot asks a question to which the user can easily answer and whose answer can efficiently reduce the number of candidate objects even if there are occluded objects and/or objects composed of multicolor parts in the scene. It considers the characteristics of features used for object recognition such as the easiness for humans to specify them by word, thus generating a user-friendly and efficient sequence of questions. Experimental results show that the robot can detect target objects by asking the questions generated by the method.

show abstract