Using robotic exploratory procedures to learn the meaning of haptic adjectives

Chu, Vivian; McMahon, Ian T.; Riano, Lorenzo; McDonald, Craig G.; He, Quan; Perez-Tejada, Jorge Martinez; Arrigo, Michael; Fitter, Naomi T.; Nappo, John C.; Darrell, Trevor; Kuchenbecker, Katherine J.

doi:10.1109/icra.2013.6631000

Cited by 94 publications

(90 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It has been especially successful in grounding concepts [9] and objects [42,31] through human-robot interaction. Works like Chu et al [11] look at mapping from text to haptic signals. Kulick et al [29] consider active learning for teaching robot to ground relational symbols.…”

Section: Related Workmentioning

confidence: 99%

Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions

Misra

Sung

Lee

et al. 2014

Robotics: Science and Systems X

View full text Add to dashboard Cite

Abstract-We consider performing a sequence of mobile manipulation tasks with instructions given in natural language (NL). Given a new environment, even a simple task such as of boiling water would be performed quite differently depending on the presence, location and state of the objects. We start by collecting a dataset of task descriptions in free-form natural language and the corresponding grounded task-logs of the tasks performed in an online robot simulator. We then build a library of verbenvironment-instructions that represents the possible instructions for each verb in that environment-these may or may not be valid for a different environment and task context. We present a model that takes into account the variations in natural language, and ambiguities in grounding them to robotic instructions with appropriate environment context and task constraints. Our model also handles incomplete or noisy NL instructions. Our model is based on an energy function that encodes such properties in a form isomorphic to a conditional random field. In evaluation, we show that our model produces sequences that perform the task successfully in a simulator and also significantly outperforms the state-of-the-art. We also verify by executing our output instruction sequences on a PR2 robot.

show abstract

Section: Related Workmentioning

confidence: 99%

Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions

Misra

Sung

Lee

et al. 2014

Robotics: Science and Systems X

View full text Add to dashboard Cite

show abstract

“…This feature function is denoted as time window feature function. In addition to these feature functions we also show results when using the features introduced by Chu et al [21]. Originally designed with the goal of object property learning, these features are divided into four distinct groups that try to depict not only the compliance, roughness, and thermal properties of the object, but also the correlation between the electrode data retrieved from the sensor.…”

Section: A Feature Comparisonmentioning

confidence: 99%

“…Originally designed with the goal of object property learning, these features are divided into four distinct groups that try to depict not only the compliance, roughness, and thermal properties of the object, but also the correlation between the electrode data retrieved from the sensor. For a more detailed description of these features please refer to [21].…”

Section: A Feature Comparisonmentioning

confidence: 99%

Stabilizing novel objects by learning to predict tactile slip

Veiga

Hoof

Peters³

et al. 2015

2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Abstract-During grasping and other in-hand manipulation tasks maintaining a stable grip on the object is crucial for the task's outcome. Inherently connected to grip stability is the concept of slip. Slip occurs when the contact between the fingertip and the object is partially lost, resulting in sudden undesired changes to the objects state. While several approaches for slip detection have been proposed in the literature, they frequently rely on previous knowledge of the manipulated object. This previous knowledge may be unavailable, seeing that robots operating in real-world scenarios often must interact with previously unseen objects.In our work we explore the generalization capabilities of well known supervised learning methods, using random forest classifiers to create generalizable slip predictors. We utilize these classifiers in the feedback loop of an object stabilization controller. We show that the controller can successfully stabilize previously unknown objects by predicting and counteracting slip events.

show abstract

“…Considerable research has been conducted to infer tactile object properties using robots as well. Two recent examples are Drimus et al [2] and Chu et al [3], who introduced methods to infer multiple object properties by analyzing touch sensor input during several controlled exploration procedures.…”

Section: Inroductionmentioning

confidence: 99%

Estimating object hardness with a GelSight touch sensor

Yuan

Srinivasan

Adelson³

2016

2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Abstract-Hardness sensing is a valuable capability for a robot touch sensor. We describe a novel method of hardness sensing that does not require accurate control of contact conditions. A GelSight sensor is a tactile sensor that provides high resolution tactile images, which enables a robot to infer object properties such as geometry and fine texture, as well as contact force and slip conditions. The sensor is pressed on silicone samples by a human or a robot and we measure the sample hardness only with data from the sensor, without a separate force sensor and without precise knowledge of the contact trajectory. We describe the features that show object hardness. For hemispherical objects, we develop a model to measure the sample hardness, and the estimation error is about 4% in the range of 8 Shore 00 to 45 Shore A. With this technology, a robot is able to more easily infer the hardness of the touched objects, thereby improving its object recognition as well as manipulation strategy.

show abstract

Using robotic exploratory procedures to learn the meaning of haptic adjectives

Cited by 94 publications

References 16 publications

Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions

Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions

Stabilizing novel objects by learning to predict tactile slip

Estimating object hardness with a GelSight touch sensor

Contact Info

Product

Resources

About