Interactive Perception: Leveraging Action in Perception and Perception in Action

Bohg, Jeannette; Hausman, Karol; Sankaran, Bharath; Brock, Oliver; Kragić, Danica; Schaal, Stefan; Sukhatme, Gaurav S.

doi:10.1109/tro.2017.2721939

Cited by 242 publications

(167 citation statements)

References 101 publications

Supporting

Mentioning

164

Contrasting

Unclassified

Order By: Relevance

“…There is compelling evidence that the interdependence and concurrency of different sensory streams aid perception and manipulation [9,18,31]. However, few studies have explicitly exploited this concurrency in representation learning.…”

Section: B Multimodal Representation Learningmentioning

confidence: 99%

Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks

Lee

Zhu

Srinivasan

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

265

122

View full text Add to dashboard Cite

Contact-rich manipulation tasks in unstructured environments often require both haptic and visual feedback. However, it is non-trivial to manually design a robot controller that combines modalities with very different characteristics. While deep reinforcement learning has shown success in learning control policies for high-dimensional inputs, these algorithms are generally intractable to deploy on real robots due to sample complexity. We use self-supervision to learn a compact and multimodal representation of our sensory inputs, which can then be used to improve the sample efficiency of our policy learning. We evaluate our method on a peg insertion task, generalizing over different geometry, configurations, and clearances, while being robust to external perturbations. We present results in simulation and on a real robot. * Authors have contributed equally and names are in alphabetical order.Authors are with the Department of Computer Science, Stanford University. [mishlee,yukez,krshna,pshah9,ssilvio,feifeili, animeshg,bohg]@stanford.edu. A. Garg is also at Nvidia, USA.

show abstract

Section: B Multimodal Representation Learningmentioning

confidence: 99%

Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks

Lee

Zhu

Srinivasan

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

265

122

View full text Add to dashboard Cite

show abstract

“…In classical AI and action theory, this inference is referred to as postdiction [12,13]. The interactive perception and postdiction capability allows agents to solve various problems including manipulation skill learning, object recognition, object dynamics learning and state representation learning [8].…”

Section: A Interactive Auditory Perception In Roboticsmentioning

confidence: 99%

Deep Neural Object Analysis by Interactive Auditory Exploration with a Humanoid Robot

Eppe

Kerzel

Strahl

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

We present a novel approach for interactive auditory object analysis with a humanoid robot. The robot elicits sensory information by physically shaking visually indistinguishable plastic capsules. It gathers the resulting audio signals from microphones that are embedded into the robotic ears. A neural network architecture learns from these signals to analyze properties of the contents of the containers. Specifically, we evaluate the material classification and weight prediction accuracy and demonstrate that the framework is fairly robust to acoustic real-world noise.

show abstract

“…An introduction of predictive force control and reactive control strategies in this domain is provided in recent years [21]. Interactive perception has been introduced as a potential field of study [22]. Such importance emerges from the fact that the perception can be facilitated by interaction with the environment.…”

mentioning

confidence: 99%

Automatic Grasping Using Tactile Sensing and Deep Calibration

Baghbahari

Behal

2019

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Tactile perception is an essential ability of intelligent robots in interaction with their surrounding environments. This perception as an intermediate level acts between sensation and action and has to be defined properly to generate suitable action in response to sensed data. In this paper, we propose a feedback approach to address robot grasping task using forcetorque tactile sensing. While visual perception is an essential part for gross reaching, constant utilization of this sensing modality can negatively affect the grasping process with overwhelming computation. In such case, human being utilizes tactile sensing to interact with objects. Inspired by, the proposed approach is presented and evaluated on a real robot to demonstrate the effectiveness of the suggested framework. Moreover, we utilize a deep learning framework called Deep Calibration in order to eliminate the effect of bias in the collected data from the robot sensors.

show abstract

Interactive Perception: Leveraging Action in Perception and Perception in Action

Cited by 242 publications

References 101 publications

Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks

Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks

Deep Neural Object Analysis by Interactive Auditory Exploration with a Humanoid Robot

Automatic Grasping Using Tactile Sensing and Deep Calibration

Contact Info

Product

Resources

About