Learning to Segment Generic Handheld Objects Using Class-Agnostic Deep Comparison and Segmentation Network

Chaudhary, Krishneel; Wada, Kazumi; Chen, Xiangyu; Kimura, Kohei; Okada, Kei; Inaba, Masayuki

doi:10.1109/lra.2018.2856917

Cited by 8 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[31] proposed a novel framework, multitask template matching (MTTM), that utilizes the nearest template of a target object from a depth image while predicting segmentation masks and a pose transformation between the template and a detected object. [32] utilized the RGB-D template of the target object, and a class-agnostic deep comparison and segmentation network was designed to output the mask and likelihood score. [33] adopted a similar methodology, which proposed a multi-stream deep similarity learning network to learn a similarity comparison model to verify the potential region.…”

Section: B Template-based Object Detectionmentioning

confidence: 99%

Template-Based Category-Agnostic Instance Detection for Robotic Manipulation

Tan

Zhou

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

An intelligent robotic system is one of the key pillars of a smart factory that requires flexibility to handle a variety of tasks. Perception is a key enabling technology for robots. Most existing object detection studies have mainly focused on category-specific objects and have achieved impressive performance. However, robotic systems, particularly in industrial scenarios, typically interact with many category-agnostic objects, which the robot must detect instantly without pre-training. Therefore, in this study, we proposed a template-based detection and segmentation approach, which incorporated a multi-level correlation model and a similarity-refine module, for handling the category-agnostic instance. The proposed approach was then validated and demonstrated in an interactive and adaptive robotic application scenario designed for the typical pick-and-place task. Among them, the picking scan path and location were instructed through human guidance with hand tracking. The neural rendering technology was also introduced to render novel views of the template. The proposed approach was evaluated using a benchmark and verified through a real demonstration.

show abstract

Section: B Template-based Object Detectionmentioning

confidence: 99%

Template-Based Category-Agnostic Instance Detection for Robotic Manipulation

Tan

Zhou

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…For example, we know "hand is part of body part" and "cup is a type of tool." Second, we assume reliable object detection with an accurate position tracking algorithm is available since there are many mature object detection algorithms [34] and stereo cameras like ZED can provide an accurate 3D position for objects [35].…”

Section: Problem Formulationmentioning

confidence: 99%

Specification mining and automated task planning for autonomous robots based on a graph-based spatial temporal logic

Liu,

Jiang,

Lin

2020

Preprint

View full text Add to dashboard Cite

We aim to enable an autonomous robot to learn new skills from demo videos and use these newly learned skills to accomplish nontrivial high-level tasks. The goal of developing such autonomous robot involves knowledge representation, specification mining, and automated task planning. For knowledge representation, we use a graph-based spatial temporal logic (GSTL) to capture spatial and temporal information of related skills demonstrated by demo videos. We design a specification mining algorithm to generate a set of parametric GSTL formulas from demo videos by inductively constructing spatial terms and temporal formulas. The resulting parametric GSTL formulas from specification mining serve as a domain theory, which is used in automated task planning for autonomous robots. We propose an automatic task planning based on GSTL where a proposer is used to generate ordered actions, and a verifier is used to generate executable task plans. A table setting example is used throughout the paper to illustrate the main ideas.

show abstract

“…Chaudhary et al [4] address part of this problem by developing a technique to segment generic hand-held objects from RBG-D input. These segmentations can then be used during long-term operation as input into an online object learning approach.…”

Section: B Perceptionmentioning

confidence: 99%

Introduction to the Special Issue on AI for Long-Term Autonomy

Kunze

Hawes

Duckett

et al. 2018

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

I. INTRODUCTION Autonomous systems have a long history in the fields of Artificial Intelligence (AI) and Robotics. However, only through recent advances in technology has it been possible to create autonomous systems capable of operating in longterm, real-world scenarios. Examples include autonomous robots that operate outdoors on land, in air, water, and space; and indoors in offices, care homes, and factories. Designing, developing, and maintaining intelligent autonomous systems that operate in real-world environments over long periods of time, i.e. weeks, months, or years, poses many challenges. This special issue focuses on such challenges and on ways to overcome them using methods from AI. Long-term autonomy can be viewed as both a challenge and an opportunity. The challenge of long-term autonomy requires system designers to ensure that an autonomous system can continue operating successfully according to its real-world application demands in unstructured and semistructured environments. This means addressing issues related to hardware and software robustness (e.g., gluing in screws and profiling for memory leaks), as well as ensuring that all modules and functions of the system can deal with the variation in the environment and tasks that is expected to occur over its operating time. Early research in longterm autonomy for mobile robots focussed extensively on the problem of coping with environment variation, e.g., performing visual localisation over seasonal changes, or SLAM in a dynamic environment. Once such challenges are overcome, the long-term operation of an autonomous system provides the opportunity to specialise that system to its tasks and operating environment, potentially improving both its general robustness and more specific task performance. This specialisation may come through the use of extensive system logs to detect and fix bugs, or through automatic online adaptation via machine learning. There is also the opportunity to aid development through the use of logs to create tests and simulations which are representative of a specific system deployment. A great many research fields have the potential to contribute to enabling and improving long-term operation of autonomous robots. For example, formal methods can be used to verify robot software or control policies to ensure safe long-term operation, and novel locomotion methods can be used to minimise a robot's long-term energy usage. However, we chose to focus on AI techniques for long-term

show abstract

Learning to Segment Generic Handheld Objects Using Class-Agnostic Deep Comparison and Segmentation Network

Cited by 8 publications

References 18 publications

Template-Based Category-Agnostic Instance Detection for Robotic Manipulation

Template-Based Category-Agnostic Instance Detection for Robotic Manipulation

Specification mining and automated task planning for autonomous robots based on a graph-based spatial temporal logic

Introduction to the Special Issue on AI for Long-Term Autonomy

Contact Info

Product

Resources

About