MRPB 1.0: A Unified Benchmark for the Evaluation of Mobile Robot Local Planning Approaches

Wen, Jian; Zhang, Xuebo; Bi, Qingchen; Pan, Zhangchao; Feng, Yanghe; Yuan, Jing

doi:10.1109/icra48506.2021.9561901

Cited by 29 publications

(25 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…however, there are limitations to the platform as only a specific number of worlds and robots can be selected. Other benchmarks that focus on navigation in social situations include Socialgym by Holtz et al [17] or HuNavSim by Perez-Higueras et al [18] MRPB 1.0 by Wen et al [19] allows researchers to benchmark local planning approaches in diverse, complex, and dynamic environments within the Gazebo simulator. Other platforms for navigating in complex dynamic and cluttered environments were presented in [20] , [21], or [22].…”

Section: Related Workmentioning

confidence: 99%

Arena-Rosnav 2.0: A Development and Benchmarking Platform for Robot Navigation in Highly Dynamic Environments

Kästner¹,

Carstens²,

Zeng³

et al. 2023

Preprint

View full text Add to dashboard Cite

Following up on our previous works, in this paper, we present Arena-Rosnav 2.0 an extension to our previous works Arena- Bench [1] and Arena-Rosnav [2], which adds a variety of additional modules for developing and benchmarking robotic navigation approaches. The platform is fundamentally restructured and provides unified APIs to add additional functionalities such as planning algorithms, simulators, or evaluation functionalities. We have included more realistic simulation and pedestrian behavior and provide a profound documentation to lower the entry barrier. We evaluated our system by first, conducting a user study in which we asked experienced researchers as well as new practitioners and students to test our system. The feedback was mostly positive and a high number of participants are utilizing our system for other research endeavors. Finally, we demonstrate the feasibility of our system by integrating two new simulators and a variety of state of the art navigation approaches and benchmark them against one another. The platform is openly available at https://github.com/Arena-Rosnav.

show abstract

Section: Related Workmentioning

confidence: 99%

Arena-Rosnav 2.0: A Development and Benchmarking Platform for Robot Navigation in Highly Dynamic Environments

Kästner¹,

Carstens²,

Zeng³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Complementariamente a la planificación global, la planificación local es clave para la evasión de obstáculos y lograr la navegación autónoma. MRPB 1.0 es un benchmark de planificación local propuesto por (Wen et al, 2021). Dicho software facilita la comparación de algoritmos diseñados por usuarios con otros de referencia ya conocidos.…”

Section: Benchmarks En Entornos Simuladosunclassified

Criterios de desempeño para evaluar algoritmos de navegación de robots móviles: una revisión

Ceballos

Suarez-Rivera

2022

Rev. iberoam. autom. inform. ind.

View full text Add to dashboard Cite

En este artículo se presenta una revisión de literatura sobre criterios de desempeño para evaluar la navegación de un robot móvil, los cuales ayudan a comparar cuantitativamente diferentes características, como: el sistema de control, la navegación en diferentes entornos de trabajo, el desempeño energético, etc. El interés en criterios de desempeño y procedimiento de comparación (benchmarks) ha crecido en los últimos años, principalmente por investigadores y fabricantes de robots que buscan satisfacer la creciente demanda de aplicaciones en el mercado global, cada vez más competido. El conjunto de criterios está compuesto por métricas, índices, mediciones y benchmarks, desde el más básico como contabilizar el éxito en alcanzar la meta, pasando por otros más elaborados como los de seguridad en la trayectoria generada en la evasión de obstáculos, hasta criterios que comparan aspectos más complejos de la navegación como el consumo energético. Finalmente, se describen algunos benchmarks y software para simulación y comparación de algoritmos de navegación. Estos criterios se constituyen en una importante herramienta para diseñadores e investigadores en robótica móvil.

show abstract

“…Existing benchmarks for robot navigation algorithms mostly focus on static environments, but few exist that cover both dynamic and static ones. Moreover, existing benchmarks for navigation in dynamic environments often contain and compare only a small number of planners [8], [9], [10]. On that account, 1 Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany d.kaestner@tu-berlin.de Fig.…”

Section: Introductionmentioning

confidence: 99%

“…Subsequently, the gathered data can be used to evaluate in terms of several performance metrics. In a recent paper by Wen et al, [8], a similar ROS and Gazebo-based benchmarking platform under the name MRPB 1.0 was proposed. It allows researchers to compare their way of local planning against several existing state-of-theart approaches.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Arena-Bench: A Benchmarking Suite for Obstacle Avoidance Approaches in Highly Dynamic Environments

Kästner

Bhuiyan

Lê

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

The ability to autonomously navigate safely, especially within dynamic environments, is paramount for mobile robotics. In recent years, DRL approaches have shown superior performance in dynamic obstacle avoidance. However, these learning-based approaches are often developed in specially designed simulation environments and are hard to test against conventional planning approaches. Furthermore, the integration and deployment of these approaches into real robotic platforms are not yet completely solved. In this paper, we present Arenabench, a benchmark suite to train, test, and evaluate navigation planners on different robotic platforms within 3D environments. It provides tools to design and generate highly dynamic evaluation worlds, scenarios, and tasks for autonomous navigation and is fully integrated into the robot operating system. To demonstrate the functionalities of our suite, we trained a DRL agent on our platform and compared it against a variety of existing different model-based and learning-based navigation approaches on a variety of relevant metrics. Finally, we deployed the approaches towards real robots and demonstrated the reproducibility of the results. The code is publicly available at github.com/ignc-research/arena-bench.

show abstract

MRPB 1.0: A Unified Benchmark for the Evaluation of Mobile Robot Local Planning Approaches

Cited by 29 publications

References 17 publications

Arena-Rosnav 2.0: A Development and Benchmarking Platform for Robot Navigation in Highly Dynamic Environments

Arena-Rosnav 2.0: A Development and Benchmarking Platform for Robot Navigation in Highly Dynamic Environments

Criterios de desempeño para evaluar algoritmos de navegación de robots móviles: una revisión

Arena-Bench: A Benchmarking Suite for Obstacle Avoidance Approaches in Highly Dynamic Environments

Contact Info

Product

Resources

About