Associating grasp configurations with hierarchical features in convolutional neural networks

Ku, Li Yang; Learned-Miller, Erik; Grupen, Roderic A.

doi:10.1109/iros.2017.8206059

Cited by 7 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, more elaborate systems for action similarity evaluation and action knowledge transfer could be introduced for an extended ADT treatment. Using deep neural network features for sub-symbolic information representation as, for example, introduced by Ku et al (2017a,b) may be promising.…”

Section: Discussionmentioning

confidence: 99%

Cut & recombine: reuse of robot action components based on simple language instructions

Tamošiūnaitė

Aein

Braun

et al. 2019

The International Journal of Robotics Research

View full text Add to dashboard Cite

Human beings can generalize from one action to similar ones. Robots cannot do this and progress concerning information transfer between robotic actions is slow. We have designed a system that performs action generalization for manipulation actions in different scenarios. It relies on an action representation for which we perform code-snippet replacement, combining information from different actions to form new ones. The system interprets human instructions via a parser using simplified language. It uses action and object names to index action data tables (ADTs), where execution-relevant information is stored. We have created an ADT database from three different sources (KUKA LWR, UR5, and simulation) and show how a new ADT is generated by cutting and recombining data from existing ADTs. To achieve this, a small set of action templates is used. After parsing a new instruction, index-based searching finds similar ADTs in the database. Then the action template of the new action is matched against the information in the similar ADTs. Code snippets are extracted and ranked according to matching quality. The new ADT is created by concatenating code snippets from best matches. For execution, only coordinate transforms are needed to account for the poses of the objects in the new scene. The system was evaluated, without additional error correction, using 45 unknown objects in 81 new action executions, with 80% success. We then extended the method including more detailed shape information, which further reduced errors. This demonstrates that cut & recombine is a viable approach for action generalization in service robotic applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Cut & recombine: reuse of robot action components based on simple language instructions

Tamošiūnaitė

Aein

Braun

et al. 2019

The International Journal of Robotics Research

View full text Add to dashboard Cite

show abstract

“…This implies that future work should focus on finding ways to analyze and compare activations in deep nets e.g. [72], either by developing retinotopic feedback loops similar to how human vision works or other recurrent ways of learning abstractions, however, without the need for pixel-wise labeling. Further on, when creating these abstractions we need to understand to what a degree we should mimic human capabilities, as this will be a crucial component in human-robot interaction.…”

Section: Discussionmentioning

confidence: 99%

Invariant Feature Mappings for Generalizing Affordance Understanding Using Regularized Metric Learning

Hjelm¹,

Ek²,

Detry³

et al. 2019

Preprint

View full text Add to dashboard Cite

This paper presents an approach for learning invariant features for object affordance understanding. One of the major problems for a robotic agent acquiring a deeper understanding of affordances is finding sensory-grounded semantics. Being able to understand what in the representation of an object makes the object afford an action opens up for more efficient manipulation, interchange of objects that visually might not be similar, transfer learning, and robot to human communication. Our approach uses a metric learning algorithm that learns a feature transform that encourages objects that affords the same action to be close in the feature space. We regularize the learning, such that we penalize irrelevant features, allowing the agent to link what in the sensory input caused the object to afford the action. From this, we show how the agent can abstract the affordance and reason about the similarity between different affordances.

show abstract

“…Ku et al [43] used convolutional layer activations to find the anthropomorphic grasping points in images. They created a mask that represented the grasping points for the robotic index finger and the thumb.…”

Section: Typementioning

confidence: 99%

Review of Deep Learning Methods in Robotic Grasp Detection

Caldera

Rassau

Chai

2018

MTI

178

View full text Add to dashboard Cite

For robots to attain more general-purpose utility, grasping is a necessary skill to master. Such general-purpose robots may use their perception abilities to visually identify grasps for a given object. A grasp describes how a robotic end-effector can be arranged to securely grab an object and successfully lift it without slippage. Traditionally, grasp detection requires expert human knowledge to analytically form the task-specific algorithm, but this is an arduous and time-consuming approach. During the last five years, deep learning methods have enabled significant advancements in robotic vision, natural language processing, and automated driving applications. The successful results of these methods have driven robotics researchers to explore the use of deep learning methods in task-generalised robotic applications. This paper reviews the current state-of-the-art in regards to the application of deep learning methods to generalised robotic grasping and discusses how each element of the deep learning approach has improved the overall performance of robotic grasp detection. Several of the most promising approaches are evaluated and the most suitable for real-time grasp detection is identified as the one-shot detection method. The availability of suitable volumes of appropriate training data is identified as a major obstacle for effective utilisation of the deep learning approaches, and the use of transfer learning techniques is proposed as a potential mechanism to address this. Finally, current trends in the field and future potential research directions are discussed.

show abstract

Associating grasp configurations with hierarchical features in convolutional neural networks

Cited by 7 publications

References 26 publications

Cut & recombine: reuse of robot action components based on simple language instructions

Cut & recombine: reuse of robot action components based on simple language instructions

Invariant Feature Mappings for Generalizing Affordance Understanding Using Regularized Metric Learning

Review of Deep Learning Methods in Robotic Grasp Detection

Contact Info

Product

Resources

About