Simitate: A Hybrid Imitation Learning Benchmark

Memmesheimer, Raphael; Mykhalchyshyna, Ivanna; Seib, Viktor; Paulus, Dietrich

doi:10.48550/arxiv.1905.06002

Cited by 4 publications

(6 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For evaluating imitation learning systems in particular, RoboTurk [28] was a recent attempt to leverage crowd sourcing to obtain data for tasks, but because of this the system has only three tasks. Whilst RoboTurk is entirely in simulation, Simitate [29] on the other hand is a hybrid approach, where real world observations (RGB-D camera calibrated against a motion capturing system) are combined with a simulated environment for benchmarking. In contrast to RoboTurk, we do not crowd source our demonstrations, but instead rely on an infinite supply of generated demonstrations collected via motion planners.…”

Section: Related Workmentioning

confidence: 99%

RLBench: The Robot Learning Benchmark & Learning Environment

James

Arrojo

et al. 2020

IEEE Robot. Autom. Lett.

233

143

View full text Add to dashboard Cite

We present a challenging new benchmark and learning-environment for robot learning: RLBench. The benchmark features 100 completely unique, hand-designed tasks ranging in difficulty, from simple target reaching and door opening, to longer multi-stage tasks, such as opening an oven and placing a tray in it. We provide an array of both proprioceptive observations and visual observations, which include rgb, depth, and segmentation masks from an over-the-shoulder stereo camera and an eye-in-hand monocular camera. Uniquely, each task comes with an infinite supply of demos through the use of motion planners operating on a series of waypoints given during task creation time; enabling an exciting flurry of demonstration-based learning. RLBench has been designed with scalability in mind; new tasks, along with their motionplanned demos, can be easily created and then verified by a series of tools, allowing users to submit their own tasks to the RLBench task repository. This large-scale benchmark aims to accelerate progress in a number of vision-guided manipulation research areas, including: reinforcement learning, imitation learning, multi-task learning, geometric computer vision, and in particular, few-shot learning. With the benchmark's breadth of tasks and demonstrations, we propose the first large-scale fewshot challenge in robotics. We hope that the scale and diversity of RLBench offers unparalleled research opportunities in the robot learning community and beyond. Benchmarking code and videos can be found here 1 .

show abstract

Section: Related Workmentioning

confidence: 99%

RLBench: The Robot Learning Benchmark & Learning Environment

James

Arrojo

et al. 2020

IEEE Robot. Autom. Lett.

233

143

View full text Add to dashboard Cite

show abstract

“…We conducted experiments on 4 different datasets. The NTU RGB+D 120 [1], UTD-MHAD [3], ARIL [4] and the Simitate [49] dataset. These datasets contain in total 5 modalities.…”

Section: Methodsmentioning

confidence: 99%

“…We further use it for our fusion experiments. We extend these experiments with activity recognition dataset containing Wi-Fi CSI fingerprints [4] and Motion Capturing data from the Simitate [49] dataset.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Gimme Signals: Discriminative signal encoding for multimodal activity recognition

Memmesheimer

Theisen

Paulus

2020

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

We present a simple, yet effective and flexible method for action recognition supporting multiple sensor modalities. Multivariate signal sequences are encoded in an image and are then classified using a recently proposed Effi-cientNet CNN architecture. Our focus was to find an approach that generalizes well across different sensor modalities without specific adaptions while still achieving good results. We apply our method to 4 action recognition datasets containing skeleton sequences, inertial and motion capturing measurements as well as Wi-Fi fingerprints that range up to 120 action classes. Our method defines the current best CNN-based approach on the NTU RGB+D 120 dataset, lifts the state of the art on the ARIL Wi-Fi dataset by +6.78%, improves the UTD-MHAD inertial baseline by +14.43%, the UTD-MHAD skeleton baseline by +1.13% and achieves 96.11% on the Simitate motion capturing data (80/20 split). We further demonstrate experiments on both, modality fusion on a signal level and signal reduction to prevent the representation from overloading.

show abstract

“…& Policy (Entangled) Fixed RL Bench [30] Env. & Policy (Entangled) Fixed Simitate [41] Env. & Policy (Entangled) Fixed MAGICAL [50] Env.…”

Section: User Workflowmentioning

confidence: 99%

The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulation

Chan¹,

Bica²,

Hüyük³

et al. 2021

Preprint

View full text Add to dashboard Cite

Understanding decision-making in clinical environments is of paramount importance if we are to bring the strengths of machine learning to ultimately improve patient outcomes. Several factors including the availability of public data, the intrinsically offline nature of the problem, and the complexity of human decision making, has meant that the mainstream development of algorithms is often geared towards optimal performance in tasks that do not necessarily translate well into the medical regime; often overlooking more niche issues commonly associated with the area. We therefore present a new benchmarking suite designed specifically for medical sequential decision making: the Medkit-Learn(ing) Environment, a publicly available Python package providing simple and easy access to high-fidelity synthetic medical data. While providing a standardised way to compare algorithms in a realistic medical setting we employ a generating process that disentangles the policy and environment dynamics to allow for a range of customisations, thus enabling systematic evaluation of algorithms' robustness against specific challenges prevalent in healthcare.Preprint. Under review.

show abstract

Simitate: A Hybrid Imitation Learning Benchmark

Cited by 4 publications

References 39 publications

RLBench: The Robot Learning Benchmark & Learning Environment

RLBench: The Robot Learning Benchmark & Learning Environment

Gimme Signals: Discriminative signal encoding for multimodal activity recognition

The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulation

Contact Info

Product

Resources

About