DynaBARN: Benchmarking Metric Ground Navigation in Dynamic Environments

Nair, Anirudh; Jiang, F; Hou, Kun; Xu, Zifan; Li, Shuozhe; Xiao, Xuesu; Stone, Peter

doi:10.1109/ssrr56537.2022.10018758

Cited by 9 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The researchers also provide an unified API in which researchers can submit their navigation planners using an unified format. Although their initial challenge is based on static environments, their subsequent works, DynaBARN by Nair et al [24] extends this to dynamic environments. However, their platform is solely for the purpose of testing and also requires some non-intuitive setup steps in order to work properly.…”

Section: Related Workmentioning

confidence: 99%

Arena-Rosnav 2.0: A Development and Benchmarking Platform for Robot Navigation in Highly Dynamic Environments

Kästner¹,

Carstens²,

Zeng³

et al. 2023

Preprint

View full text Add to dashboard Cite

Following up on our previous works, in this paper, we present Arena-Rosnav 2.0 an extension to our previous works Arena- Bench [1] and Arena-Rosnav [2], which adds a variety of additional modules for developing and benchmarking robotic navigation approaches. The platform is fundamentally restructured and provides unified APIs to add additional functionalities such as planning algorithms, simulators, or evaluation functionalities. We have included more realistic simulation and pedestrian behavior and provide a profound documentation to lower the entry barrier. We evaluated our system by first, conducting a user study in which we asked experienced researchers as well as new practitioners and students to test our system. The feedback was mostly positive and a high number of participants are utilizing our system for other research endeavors. Finally, we demonstrate the feasibility of our system by integrating two new simulators and a variety of state of the art navigation approaches and benchmark them against one another. The platform is openly available at https://github.com/Arena-Rosnav.

show abstract

Section: Related Workmentioning

confidence: 99%

Arena-Rosnav 2.0: A Development and Benchmarking Platform for Robot Navigation in Highly Dynamic Environments

Kästner¹,

Carstens²,

Zeng³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The 3rd BARN Challenge had two phases: a qualifying phase evaluated in simulation, and a final phase evaluated in three physical obstacle courses. The qualifying phase took place before the ICRA 2024 conference using the BARN dataset [5] (with the recent addition of DynaBARN [6]), which is composed of 300 obstacle courses in Gazebo simulation randomly generated by cellular automata. The top four teams from the simulation phase were then invited to compete in three different physical obstacle courses set up by the organizers at ICRA 2024 in the PACIFICO Yokohama conference center.…”

Section: The 3rd Barn Challenge Overviewmentioning

confidence: 99%

“…Based on the first three year's BARN challenges, the organizers plan to make the following changes in the next BARN challenge in ICRA 2025. First, dynamic obstacles will be introduced to the currently static obstacle courses [6], [51]. For the first competition with dynamic obstacles, the organizers will allow collisions with dynamic obstacles and only add a penalty, whereas collisions with static obstacles will still be counted as a total failure.…”

Section: Future Plansmentioning

confidence: 99%

See 1 more Smart Citation

Autonomous Ground Navigation in Highly Constrained Spaces: Lessons Learned From the Benchmark Autonomous Robot Navigation Challenge at ICRA 2022 [Competitions]

Xiao

Wang

et al. 2022

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

The 3rd BARN (Benchmark Autonomous Robot Navigation) Challenge took place at the 2024 IEEE International Conference on Robotics and Automation (ICRA 2024) in Yokohama, Japan and continued to evaluate the performance of state-of-the-art autonomous ground navigation systems in highly constrained environments. Similar to the trend in The 1st and 2nd BARN Challenge at ICRA 2022 and 2023 in Philadelphia (North America) and London (Europe), The 3rd BARN Challenge in Yokohama (Asia) became more regional, i.e., mostly Asian teams participated. The size of the competition has slightly shrunk (six simulation teams, four of which were invited to the physical competition). The competition results, compared to last two years, suggest that the field has adopted new machine learning approaches while at the same time slightly converged to a few common practices. However, the regional nature of the physical participants suggests a challenge to promote wider participation all over the world and provide more resources to travel to the venue. In this article, we discuss the challenge, the approaches used by the three winning teams, and lessons learned to direct future research and competitions.

show abstract

“…There has been much recent development of open-source robotics libraries and associated benchmarks demonstrat- ing their performance as well as a plethora of workshops and tutorials focusing on benchmarking robotics applications [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22]. However, most of these robotics benchmarks focus on algorithm correctness (functional testing) in the context of domain specific problems, as well as end-to-end latency on CPUs [31], [32], [33], [34], [37], [35], [38], [36], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48]. A few works also analyze some non-functional metrics, such as CPU performance benchmarks, to explore bottleneck behaviors in selected workloads [23], [24], [49].…”

Section: B Robotics Benchmarksmentioning

confidence: 99%

RobotCore: An Open Architecture for Hardware Acceleration in ROS 2

Mayoral-Vilches¹,

Neuman

Plancher

et al. 2022

2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

We introduce RobotPerf, a vendor-agnostic benchmarking suite designed to evaluate robotics computing performance across a diverse range of hardware platforms using ROS 2 as its common baseline. The suite encompasses ROS 2 packages covering the full robotics pipeline and integrates two distinct benchmarking approaches: black-box testing, which measures performance by eliminating upper layers and replacing them with a test application, and grey-box testing, an application-specific measure that observes internal system states with minimal interference. Our benchmarking framework provides ready-to-use tools and is easily adaptable for the assessment of custom ROS 2 computational graphs. Drawing from the knowledge of leading robot architects and system architecture experts, RobotPerf establishes a standardized approach to robotics benchmarking. As an open-source initiative, RobotPerf remains committed to evolving with community input to advance the future of hardware-accelerated robotics.

show abstract

DynaBARN: Benchmarking Metric Ground Navigation in Dynamic Environments

Cited by 9 publications

References 31 publications

Arena-Rosnav 2.0: A Development and Benchmarking Platform for Robot Navigation in Highly Dynamic Environments

Arena-Rosnav 2.0: A Development and Benchmarking Platform for Robot Navigation in Highly Dynamic Environments

Autonomous Ground Navigation in Highly Constrained Spaces: Lessons Learned From the Benchmark Autonomous Robot Navigation Challenge at ICRA 2022 [Competitions]

RobotCore: An Open Architecture for Hardware Acceleration in ROS 2

Contact Info

Product

Resources

About