Robotic Pick and Assembly Using Deep Learning and Hybrid Vision/Force Control

Almaghout, Karam; Boby, Riby Abraham; Othman, Mostafa; Abdelaziz, Shaarawy,; Klimchik, Alexandr

doi:10.1109/nir52917.2021.9666138

Cited by 3 publications

(1 citation statement)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Perception of DLOs includes recognition, state estimation, and tracking using sensory systems like vision, force, tactile, and others [7,8]. Vision sensors are widely used for perception in the robotic manipulation since they are affordable and proper for tasks that include objects detection and position localization for both rigid and deformable objects [9,10]. In the visionbased robotic manipulation of DLOs, a group of researchers considers using add-on markers as feature points and manipulating these points [11][12][13].…”

Section: Introductionmentioning

confidence: 99%

Vision-Based Robotic Comanipulation for Deforming Cables

Almaghout¹,

Klimchik²

2022

Nelin. Dinam.

View full text Add to dashboard Cite

Although deformable linear objects (DLOs), such as cables, are widely used in the majority of life fields and activities, the robotic manipulation of these objects is considerably more complex compared to the rigid-body manipulation and still an open challenge. In this paper, we introduce a new framework using two robotic arms cooperatively manipulating a DLO from an initial shape to a desired one. Based on visual servoing and computer vision techniques, a perception approach is proposed to detect and sample the DLO as a set of virtual feature points. Then a manipulation planning approach is introduced to map between the motion of the manipulators end effectors and the DLO points by a Jacobian matrix. To avoid excessive stretching of the DLO, the planning approach generates a path for each DLO point forming profiles between the initial and desired shapes. It is guaranteed that all these intershape profiles are reachable and maintain the cable length constraint. The framework and the aforementioned approaches are validated in real-life experiments.

show abstract

Section: Introductionmentioning

confidence: 99%

Vision-Based Robotic Comanipulation for Deforming Cables

Almaghout¹,

Klimchik²

2022

Nelin. Dinam.

View full text Add to dashboard Cite

show abstract

ForceSight: Text-Guided Mobile Manipulation with Visual-Force Goals

Collins,

Houff,

Tan

et al. 2024

2024 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

6D Robotic Assembly Based on RGB-only Object Pose Estimation

Kun

Lian

et al. 2022

2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Comprehending natural language instructions is a critical skill for robots to cooperate effectively with humans. In this paper, we aim to learn 6D poses for robotic assembly by natural language instructions. For this purpose, Language-Instructed 6D Pose Regression Network (LanPose) is proposed to jointly predict the 6D poses of the observed object and the corresponding assembly position. Our proposed approach is based on the fusion of geometric and linguistic features, which allows us to finely integrate multi-modality input and map it to the 6D pose in SE(3) space by the cross-attention mechanism and the language-integrated 6D pose mapping module, respectively. To validate the effectiveness of our approach, an integrated robotic system is established to precisely and robustly perceive, grasp, manipulate and assemble blocks by language commands. 98.09 and 93.55 in ADD(-S)-0.1d are derived for the prediction of 6D object pose and 6D assembly pose, respectively. Both quantitative and qualitative results demonstrate the effectiveness of our proposed languageinstructed 6D pose estimation methodology and its potential to enable robots to better understand and execute natural language instructions.

show abstract

Robotic Pick and Assembly Using Deep Learning and Hybrid Vision/Force Control

Cited by 3 publications

References 21 publications

Vision-Based Robotic Comanipulation for Deforming Cables

Vision-Based Robotic Comanipulation for Deforming Cables

ForceSight: Text-Guided Mobile Manipulation with Visual-Force Goals

6D Robotic Assembly Based on RGB-only Object Pose Estimation

Contact Info

Product

Resources

About