Autonomous acquisition of multimodal information for online object concept formation by robots

Araki, Takaya; Nagaoka, Tomoaki; Nagai, Takayuki; Funakoshi, Kotaro; Nakano, Mikio; Iwahashi, Naoto

doi:10.1109/iros.2011.6094814

Cited by 16 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Araki [44] proposes a multimodal approach (including vision, sound and haptic properties of objects) that is implemented on a real robot, and focuses on learning object concepts by using Latent Dirichlet Allocation (LDA). The multimodal data are acquired autonomously by a robot equipped with a 3D visual sensor, two arms and a small handheld observation table that serves as the platform for capturing multi-view visual images of objects.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Comparison Studies on Active Cross-Situational Object-Word Learning Using Non-Negative Matrix Factorization and Latent Dirichlet Allocation

Chen

Bordes

Filliat

2018

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Abstract-Future intelligent robots are expected to be able to adapt continuously to their environment. For this purpose, recognizing new objects and learning new words through interactive learning with humans is fundamental. Such setup results in ambiguous teaching data which humans have been shown to address using cross-situational learning, i.e. by analyzing common factors between multiple learning situations. Moreover, they have been shown to be more efficient when actively choosing the learning samples, e.g. which object they want to learn. Implementing such abilities on robots can be performed by latenttopic learning models such as Non-Negative Matrix Factorization or Latent Dirichlet Allocation. These cross-situational learning methods tackle referential and linguistic ambiguities, and can be associated with active learning strategies. We propose two such methods: the Maximum Reconstruction Error based Selection (MRES) and Confidence Base Exploration (CBE). We present extensive experiments using these two learning algorithms through a systematic analysis on the effects of these active learning strategies in contrast with random choice. In addition, we study the factors underlying the active learning by focusing on the use of sample repetition, one of the learning behaviors that have been shown to be important for humans.

show abstract

Section: Related Workmentioning

confidence: 99%

“…While several models use specific learning algorithms [46], [41], [23], other use very generic topic models such as NMF [42] and LDA [44]. The latter ones provide a sound definition of the problem (i.e., finding the hidden cause that generates a visual feature and an associated word).…”

Section: Related Workmentioning

confidence: 99%

Comparison Studies on Active Cross-Situational Object-Word Learning Using Non-Negative Matrix Factorization and Latent Dirichlet Allocation

Chen

Bordes

Filliat

2018

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

show abstract

“…Unlike existing datasets, the datasets reported in this paper offer the following advantages to the interested researchers. First, some of these datasets were collected in a single (or similar) experimental setting to address particular problems such as object recognition and categorization [1], [2]. Second, compared with the datasets in [6], [5], our study offers the following advantages to the community: a high number of cameras that can record color images and the usage of sensors that are mounted on the iCub robot.…”

Section: Introductionmentioning

confidence: 99%

The iCub Multisensor Datasets for Robot and Computer Vision Applications

Kirtay

Albanese

Vannucci

et al. 2020

Proceedings of the 2020 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Multimodal information can significantly increase the perceptual capabilities of robotic agents, at the cost of a more complex sensory processing. This complexity can be reduced by employing machine learning techniques, provided that there is enough meaningful data to train on. This paper reports on creating novel datasets constructed by employing the iCub robot equipped with an additional depth sensor and color camera. We used the robot to acquire color and depth information for 210 objects in different acquisition scenarios. At the end, the results were large scale datasets that can be used for robot and computer vision applications: multisensory object representation, action recognition, rotation and distance invariant object recognition. CCS CONCEPTS • Computing methodologies → Neural networks.

show abstract

“…Other studies, e.g. [61][62][63][64], have presented solutions for merging different sensors' data to address classificationtype problems, such as object/gesture recognition or speaker identification/spatial localisation, rather than imitating demonstrated behaviours. Different approaches have been used to solve imitation learning using different sensor informations, such as hierarchical architectures based on multiple internal models [46,[57][58][59], and Gaussian Mixture Regression together with Hidden Markov Model [60].…”

Section: Introductionmentioning

confidence: 99%

Online Multimodal Ensemble Learning Using Self-Learned Sensorimotor Representations

Zambelli

Demiris

2017

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Internal models play a key role in cognitive agents by providing on the one hand predictions of sensory consequences of motor commands (forward models), and on the other hand inverse mappings (inverse models) to realise tasks involving control loops, such as imitation tasks. The ability to predict and generate new actions in continuously evolving environments intrinsically requiring the use of different sensory modalities is particularly relevant for autonomous robots, which must also be able to adapt their models online. We present a learning architecture based on self-learned multimodal sensorimotor representations. To attain accurate forward models, we propose an online heterogeneous ensemble learning method that allows us to improve the prediction accuracy by leveraging differences of multiple diverse predictors. We further propose a method to learn inverse models on-the-fly to equip a robot with multimodal learning skills to perform imitation tasks using multiple sensory modalities. We have evaluated the proposed methods on an iCub humanoid robot. Since no assumptions are made on the robot kinematic/dynamic structure, the method can be applied to different robotic platforms.

show abstract

Autonomous acquisition of multimodal information for online object concept formation by robots

Cited by 16 publications

References 13 publications

Comparison Studies on Active Cross-Situational Object-Word Learning Using Non-Negative Matrix Factorization and Latent Dirichlet Allocation

Comparison Studies on Active Cross-Situational Object-Word Learning Using Non-Negative Matrix Factorization and Latent Dirichlet Allocation

The iCub Multisensor Datasets for Robot and Computer Vision Applications

Online Multimodal Ensemble Learning Using Self-Learned Sensorimotor Representations

Contact Info

Product

Resources

About