Kinematically-Informed Interactive Perception: Robot-Generated 3D Models for Classification

Griffin, Brent; Corso, Jason J.

doi:10.48550/arxiv.1901.05580

Cited by 3 publications

(3 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Venkataraman et al tackled the challenge of generating generic 3D models for original items using a robot capable of decluttering items to enhance organization [29]. Their approach involved creating models of grasped objects through simultaneous manipulation and tracking.…”

Section: Related Workmentioning

confidence: 99%

Design of the Integrated Cognitive Perception Model for Developing Situation-Awareness of an Autonomous Smart Agent

DAĞLARLI

2023

Balkan Journal of Electrical and Computer Engineering

View full text Add to dashboard Cite

This study explores the potential for autonomous agents to develop environmental awareness through perceptual attention. The main objective is to design a perception system architecture that mimics human-like perception, enabling smart agents to establish effective communication with humans and their surroundings. Overcoming the challenges of modeling the agent's environment and addressing the coordination issues of multi-modal perceptual stimuli is crucial for achieving this goal. Existing research falls short in meeting these requirements, prompting the introduction of a novel solution: a cognitive multi-modal integrated perception system. This computational framework incorporates fundamental feature extraction, recognition tasks, and spatial-temporal inference while facilitating the modeling of perceptual attention and awareness. To evaluate its performance, experimental tests and verification are conducted using a software framework integrated into a sandbox game platform. The model's effectiveness is assessed through a simple interaction scenario. The study's results demonstrate the successful validation of the proposed research questions.

show abstract

Section: Related Workmentioning

confidence: 99%

Design of the Integrated Cognitive Perception Model for Developing Situation-Awareness of an Autonomous Smart Agent

DAĞLARLI

2023

Balkan Journal of Electrical and Computer Engineering

View full text Add to dashboard Cite

show abstract

“…Similarly, modern semi-supervised video segmentation [3], [4] will be limited by the quality of the initial masks, placing increasing importance on the quality of a single-frame segmentation. Instance segmentation in 3D is even more challenging, requiring either multiview mapping or volumetric shape completion, which are still active areas of research [5], [6], [7], [8], [9].…”

Section: Introductionmentioning

confidence: 99%

Fusing RGBD Tracking and Segmentation Tree Sampling for Multi-Hypothesis Volumetric Segmentation

Price¹,

Huang²,

Berenson³

2021

Preprint

View full text Add to dashboard Cite

Despite rapid progress in scene segmentation in recent years, 3D segmentation methods are still limited when there is severe occlusion. The key challenge is estimating the segment boundaries of (partially) occluded objects, which are inherently ambiguous when considering only a single frame. In this work, we propose Multihypothesis Segmentation Tracking (MST), a novel method for volumetric segmentation in changing scenes, which allows scene ambiguity to be tracked and our estimates to be adjusted over time as we interact with the scene. Two main innovations allow us to tackle this difficult problem: 1) A novel way to sample possible segmentations from a segmentation tree; and 2) A novel approach to fusing tracking results with multiple segmentation estimates. These methods allow MST to track the segmentation state over time and incorporate new information, such as new objects being revealed. We evaluate our method on several cluttered tabletop environments in simulation and reality. Our results show that MST outperforms baselines in all tested scenes.

show abstract

“…works in which robots grasp unknown objects in order to learn their visual appearance [6]- [11]. Most of these methods predate deep learning and require a human to design a visual model or other visual heuristics for recognizing the robot manipulator.…”

Section: Introductionmentioning

confidence: 99%

Robot-Supervised Learning for Object Segmentation

Florence¹,

Corso²,

Griffin³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

To be effective in unstructured and changing environments, robots must learn to recognize new objects. Deep learning has enabled rapid progress for object detection and segmentation in computer vision; however, this progress comes at the price of human annotators labeling many training examples. This paper addresses the problem of extending learning-based segmentation methods to robotics applications where annotated training data is not available. Our method enables pixelwise segmentation of grasped objects. We factor the problem of segmenting the object from the background into two sub-problems: (1) segmenting the robot manipulator and object from the background and (2) segmenting the object from the manipulator. We propose a kinematics-based foreground segmentation technique to solve (1). To solve (2), we train a self-recognition network that segments the robot manipulator. We train this network without human supervision, leveraging our foreground segmentation technique from (1) to label a training set of images containing the robot manipulator without a grasped object. We demonstrate experimentally that our method outperforms state-of-the-art adaptable in-hand object segmentation. We also show that a training set composed of automatically labelled images of grasped objects improves segmentation performance on a test set of images of the same objects in the environment.

show abstract

Kinematically-Informed Interactive Perception: Robot-Generated 3D Models for Classification

Cited by 3 publications

References 30 publications

Design of the Integrated Cognitive Perception Model for Developing Situation-Awareness of an Autonomous Smart Agent

Design of the Integrated Cognitive Perception Model for Developing Situation-Awareness of an Autonomous Smart Agent

Fusing RGBD Tracking and Segmentation Tree Sampling for Multi-Hypothesis Volumetric Segmentation

Robot-Supervised Learning for Object Segmentation

Contact Info

Product

Resources

About