OCRTOC: A Cloud-Based Competition and Benchmark for Robotic Grasping and Manipulation

Liu, Ziyuan; Liu, Wei; Qin, Yuzhe; Xiang, Fanbo; Gou, Minghao; Xin, Songyan; Roa, Máximo A.; Çallı, Berk; Su, Hao; Sun, Yu; Tan, Ping

doi:10.1109/lra.2021.3129136

Cited by 36 publications

(15 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5. Beyond that, experiments also include YCB objects [4] obtained from [20] 1: Success rate across scenarios. For tasks with rectangular prisms and cylinders, a total of 450 trials are run for each algorithm.…”

Section: Methodsmentioning

confidence: 99%

Safe, Occlusion-Aware Manipulation for Online Object Reconstruction in Confined Spaces

Miao¹,

Wang²,

Bekris³

2022

Preprint

View full text Add to dashboard Cite

Recent work in robotic manipulation focuses on object retrieval in cluttered space under occlusion. Nevertheless, the majority of efforts lack an analysis of conditions for the completeness of the approaches or the methods apply only when objects can be removed from the workspace. This work formulates the general, occlusion-aware manipulation task, and focuses on safe object reconstruction in a confined space with in-place relocation. A framework that ensures safety with completeness guarantees is proposed. Furthermore, an algorithm, which is an instantiation of this framework for monotone instances, is developed and evaluated empirically by comparing against a random and a greedy baseline on randomly generated experiments in simulation. Even for cluttered scenes with realistic objects, the proposed algorithm significantly outperforms the baselines and maintains a high success rate across experimental conditions.

show abstract

Section: Methodsmentioning

confidence: 99%

Safe, Occlusion-Aware Manipulation for Online Object Reconstruction in Confined Spaces

Miao¹,

Wang²,

Bekris³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…All object for packing in our experiments come from the YCB dataset [41] and OCRTOC dataset [42]. We select 121 types of objects to construct our training and test dataset, as representative objects are shown in Fig.…”

Section: A Implementation Detailsmentioning

confidence: 99%

Planning Irregular Object Packing via Hierarchical Reinforcement Learning

Huang

Wang

Zhou

et al. 2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Object packing by autonomous robots is an important challenge in warehouses and logistics industry. Most conventional data-driven packing planning approaches focus on regular cuboid packing, which are usually heuristic and limit the practical use in realistic applications with everyday objects. In this paper, we propose a deep hierarchical reinforcement learning approach to simultaneously plan packing sequence and placement for irregular object packing. Specifically, the top manager network infers packing sequence from six principal view heightmaps of all objects, and then the bottom worker network receives heightmaps of the next object to predict the placement position and orientation. The two networks are trained hierarchically in a self-supervised Q-Learning framework, where the rewards are provided by the packing results based on the top height , object volume and placement stability in the box. The framework repeats sequence and placement planning iteratively until all objects have been packed into the box or no space is remained for unpacked items. We compare our approach with existing robotic packing methods for irregular objects in a physics simulator. Experiments show that our approach can pack more objects with less time cost than the state-of-the-art packing methods of irregular objects. We also implement our packing plan with a robotic manipulator to show the generalization ability in the real world.

show abstract

“…The advancement of affordable consumer-grade and precise 3D scanner hardware (SHINING 3D EinScan-SP) allows to generate custom 3D models for individual use-cases. For our work we chose a subset of 33 high-quality meshes from [3] being part of YCB object set [13], 4 from [15], scanned 36 objects by ourselves and remodeled 9 in CAD software when scanning was not possible. Obtaining accurate 3D models for all objects is challenging and time consuming.…”

Section: A Custom Object Dataset and Novel Object Testsetmentioning

confidence: 99%

“…Besides this limitation, order picking systems usually depend on an additional upstream object detection. Existing datasets containing textured objects [13], [15], [12] are often limited to the household domain, available in small numbers and represent only a small subset of possible objects in warehouse or industry settings or do not contain real world scans [10].…”

Section: Introductionmentioning

confidence: 99%

MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis

Gilles¹,

Chen²,

Winter³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous bin picking poses significant challenges to vision-driven robotic systems given the complexity of the problem, ranging from various sensor modalities, to highly entangled object layouts, to diverse item properties and gripper types. Existing methods often address the problem from one perspective. Diverse items and complex bin scenes require diverse picking strategies together with advanced reasoning. As such, to build robust and effective machine-learning algorithms for solving this complex task requires significant amounts of comprehensive and high quality data. Collecting such data in real world would be too expensive and time prohibitive and therefore intractable from a scalability perspective. To tackle this big, diverse data problem, we take inspiration from the recent rise in the concept of metaverses, and introduce MetaGraspNet, a large-scale photo-realistic bin picking dataset constructed via physics-based metaverse synthesis. The proposed dataset contains 217k RGBD images across 82 different article types, with full annotations for object detection, amodal perception, keypoint detection, manipulation order and ambidextrous grasp labels for a parallel-jaw and vacuum gripper. We also provide a real dataset consisting of over 2.3k fully annotated high-quality RGBD images, divided into 5 levels of difficulties and an unseen object set to evaluate different object and layout properties. Finally, we conduct extensive experiments showing that our proposed vacuum seal model and synthetic dataset achieves state-of-the-art performance and generalizes to real world use-cases.

show abstract

OCRTOC: A Cloud-Based Competition and Benchmark for Robotic Grasping and Manipulation

Cited by 36 publications

References 32 publications

Safe, Occlusion-Aware Manipulation for Online Object Reconstruction in Confined Spaces

Safe, Occlusion-Aware Manipulation for Online Object Reconstruction in Confined Spaces

Planning Irregular Object Packing via Hierarchical Reinforcement Learning

MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis

Contact Info

Product

Resources

About