SwarmRob: A Docker-Based Toolkit for Reproducibility and Sharing of Experimental Artifacts in Robotics Research

Abstract

Due to the complexity of robotics, the reproducibility of results and experiments is one of the fundamental problems in robotics research. While the problem has been identified by the community, the approaches that address the problem appropriately are limited. The toolkit proposed in this paper tries to deal with the problem of reproducibility and sharing of experimental artifacts in robotics research by a holistic approach based on operating-system-level virtualization. The experimental artifacts of an experiment are isolated in "containers" that can be distributed to other researchers. Based on this, this paper presents a novel experimental workflow to describe, execute and distribute experimental software-artifacts to heterogeneous robots dynamically. As a result, the proposed solution supports researchers in executing and reproducing experimental evaluations.

Benchmarking Teamwork of Humans and Cobots—An Overview of Metrics, Strategies, and Tasks

“…CodeOcean, 4 IEEE DataPort 5 ). From a technical point of view, the situation may be enhanced in the future by unified architectures for experimental cobot systems [283] and by the use of containerization techniques [284], [285]. Yet, to the best of our knowledge, these approaches have not yet been applied to complex HRI user studies -we hence classify the reproducibility of human-participant studies as low.…”

State of the Art and Future Trends in Data Reduction for High-Performance Computing

“…It provides access to both local and remote data and features built-in support for a wide range of remote data sources. SwarmRob is a toolkit for making robotics research reproducible [117]. In addition to providing a toolkit, the authors propose a new workflow that is separated into research and review phases.…”

Try to Start It! The Challenge of Reusing Code in Robotics Research

“…SwarmRob is another toolkit recently proposed [37], which uses a holistic approach based on operating-system-level virtualization for dealing with the problem of reproducibility and sharing of experimental artifacts.…”

“…In this work, we extend the use of Docker to a set of representative robotics research papers, illustrating its usefulness in an academic setting. Unlike other frameworks [35], [37], the proposed method does not interfere with the common development workflow, since it smoothly integrates the authors' code repository with the automatic generation of Docker images. Moreover, it proves to be useful in a wide range of robotic domains.…”