Autonomous acquisition of multimodal information for online object concept formation by robots

Araki,; Nakamura,; Nagai,; Funakoshi,; Nakano, Mikio; Iwahashi,

doi:10.1109/iros.2011.6048422

Cited by 37 publications

(32 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to bring the object, the robot must find and recognize the object using multimodal information of speech of the object name and images of the object. Several researchers have studied learning objects through interaction by multimodal information so that robots can recognize the object [2], [3], [6], [7]. In [6], they proposed a language acquisition model using speech and image information.…”

Section: Related Workmentioning

confidence: 99%

“…In [6], they proposed a language acquisition model using speech and image information. Araki et al presented a robot that can learn the concept of an object via spoken dialog [7]. Ozasa et al proposed a method for object recognition using multimodal information in order to learn unknown objects in the object manipulation task where a human asks a robot via voice to bring an object among several objects on a table [2].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Selection of Unknown Objects Specified by Speech Using Models Constructed from Web Images

Nishimura

Ozasa

Ariki

et al. 2014

2014 22nd International Conference on Pattern Recognition

View full text Add to dashboard Cite

To select an object requested by human voice among several unknown objects is one of the important tasks for household robots that assist human's daily lives. In this paper, we propose a method that can achieve this task. Using image models which are constructed by Web images collected from the results of speech recognition, the proposed method enables a robot to select an object specified by a speech from several unknown objects in a real environment.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Selection of Unknown Objects Specified by Speech Using Models Constructed from Web Images

Nishimura

Ozasa

Ariki

et al. 2014

2014 22nd International Conference on Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…The tactile array sensor consists of 162 tactile sensors, and 162 time series sensor values are obtained by the hand grasping the object. The sensor values are approximated by using a sigmoid function, the parameters of which are used as feature vectors for the haptic information [14]. The robot grasps the object five times, and therefore, 810(= 162 × 5) sensor values are obtained from one object.…”

Section: Visual Informationmentioning

confidence: 99%

Multimodal concept and word learning using phoneme sequences with errors

Nagaoka

Araki

Nagai

et al. 2013

2013 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

In this study, we propose a method for concept formation and word acquisition for robots. The proposed method is based on multimodal latent Dirichlet allocation (MLDA) and the nested Pitman-Yor language model (NPYLM). A robot obtains haptic, visual, and auditory information by grasping, observing, and shaking an object. At the same time, a user teaches object features to the robot through speech, which is recognized using only acoustic models and transformed into phoneme sequences. As the robot is supposed to have no language model in advance, the recognized phoneme sequences include many phoneme recognition errors. Moreover, the recognized phoneme sequences with errors are segmented into words in an unsupervised manner; however, not all words are necessarily segmented correctly. The words including these errors have a negative effect on the learning of word meanings. To overcome this problem, we propose a method to improve unsupervised word segmentation and to reduce phoneme recognition errors by using multimodal object concepts. In the proposed method, object concepts are used to enhance the accuracy of word segmentation, reduce phoneme recognition errors, and correct words so as to improve the categorization accuracy. We experimentally demonstrate that the proposed method can improve the accuracy of word segmentation and reduce the phoneme recognition error and that the obtained words enhance the categorization accuracy.

show abstract

“…To obtain visual information, the robot uses a small handheld observation table with an XBee wireless controller to control the viewpoints for observing the object [9]. Considering the computational cost, Speeded-Up Robust Features (SURF) [10] is used and the descriptors with 128 dimensions are collected from each captured image and stored in the object database.…”

Section: A Superpixel-based Object Detectionmentioning

confidence: 99%

A planning method for efficient mobile manipulation considering ambiguity

Attamimi

Ito

Nagaoka

et al. 2012

2012 IEEE/RSJ International Conference on Intelligent Robots and Systems

View full text Add to dashboard Cite

In this study, we propose a system to enable robots to navigate more efficiently to target objects in order to carry out mobile manipulation tasks. Because a robot arm has limited reach, a target object cannot be grasped if the distance to the object exceeds the reach. At the same time, navigation errors may increase as the robot moves around in an environment.To mitigate these problems, we propose the use of maps that express the arm reachability and navigation reachability. Places with minimum probability of navigation errors and maximum tolerable navigation errors for the reach of the arm can be determined by integrating these two maps. We also consider the fact that the target object position is not always known. To cope with this problem, we introduce an object existence map that represents the ambiguity of the target object position, considering the past position. If the ambiguity is large, it is possible to select a place from where the object searching task can be performed with a wide angle of view by the robot. Efficient object searching can be achieved by changing the task objective from searching to grasping and vice versa. We conducted some mobile manipulation tasks to evaluate the proposed method. The results showed that the proposed system can perform such tasks effectively.

show abstract

Autonomous acquisition of multimodal information for online object concept formation by robots

Cited by 37 publications

References 0 publications

Selection of Unknown Objects Specified by Speech Using Models Constructed from Web Images

Selection of Unknown Objects Specified by Speech Using Models Constructed from Web Images

Multimodal concept and word learning using phoneme sequences with errors

A planning method for efficient mobile manipulation considering ambiguity

Contact Info

Product

Resources

About