Accurate, Low-Latency Visual Perception for Autonomous Racing: Challenges, Mechanisms, and Practical Solutions

Strobel, Kieran; Zhu, Sibo; Chang, Raphael; Koppula, Skanda

doi:10.1109/iros45743.2020.9341683

Cited by 13 publications

(8 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Racing on an unknown track pushes requirements on detection range and accuracy as these directly affect the achievable maximum speed. This is in addition to existing requirements on map quality as in previous AMZ works [2], [3], [6] and other FSD teams [7].…”

Section: Related Workmentioning

confidence: 79%

“…A key task of the presented perception system is the detection of cones. In [7], and [8] the authors present stateof-the-art CNN-based vision-only detection systems for cone detection based on versions of YOLO [9], [10]. In order to make better use of the accurate range measurements from LiDAR in combination with the rich semantic information from cameras, we implement an early sensor fusion approach for accurate color and position detection of cones.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Accurate Mapping and Planning for Autonomous Racing

Andresen

Brandemuehl

Hönger

et al. 2020

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

This paper presents the perception, mapping, and planning pipeline implemented on an autonomous race car. It was developed by the 2019 AMZ driverless team for the Formula Student Germany (FSG) 2019 driverless competition, where it won 1st place overall. The presented solution combines early fusion of camera and LiDAR data, a layered mapping approach, and a planning approach that uses Bayesian filtering to achieve high-speed driving on unknown race tracks while creating accurate maps. We benchmark the method against our team's previous solution, which won FSG 2018, and show improved accuracy when driving at the same speeds. Furthermore, the new pipeline makes it possible to reliably raise the maximum driving speed in unknown environments from 3 m/s to 12 m/s while still mapping with an acceptable RMSE of 0.29 m.

show abstract

Section: Related Workmentioning

confidence: 79%

Section: Related Workmentioning

confidence: 99%

Accurate Mapping and Planning for Autonomous Racing

Andresen

Brandemuehl

Hönger

et al. 2020

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

show abstract

“…In addition, the FSD competition provides yellow and blue cones as the race track and the teams need to detect those cones at high vehicle speeds. As a result, particular applications of YOLO-based methods are used to detect the cones (Dhall et al, 2019;Strobel et al, 2020).…”

Section: Softwarementioning

confidence: 99%

TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge

Betz¹,

Betz²,

Fent³

et al. 2022

Preprint

View full text Add to dashboard Cite

For decades, motorsport has been an incubator for innovations in the automotive sector and brought forth systems like disk brakes or rearview mirrors. Autonomous racing series such as Roborace, F1Tenth, or the Indy Autonomous Challenge (IAC) are envisioned as playing a similar role within the autonomous vehicle sector, serving as a proving ground for new technology at the limits of the autonomous systems capabilities. This paper outlines the software stack and approach of the TUM Autonomous Motorsport team for their participation in the Indy Autonomous Challenge, which holds two competitions: A single-vehicle competition on the Indianapolis Motor Speedway and a passing competition at the Las Vegas Motor Speedway. Nine university teams used an identical vehicle platform: A modified Indy Lights chassis equipped with sensors, a computing platform, and actuators. All the teams developed different algorithms for object detection, localization, planning, prediction, and control of the race cars. The team from TUM placed first in Indianapolis and secured second place in Las Vegas. During the final of the passing competition, the TUM team reached speeds and accelerations close to the limit of the vehicle, peaking at around 270 km h −1 and 28 m s −2 . This paper will present details of the vehicle hardware platform, the developed algorithms, and the workflow to test and enhance the software applied during the two-year project. We derive deep insights into the autonomous vehicle's behavior at high speed and high acceleration by providing a detailed competition analysis. Based on this, we deduce a list of lessons learned and provide insights on promising areas of future work based on the real-world evaluation of the displayed concepts.

show abstract

“…Aside from the challenges in modeling the complex dynamics, a significant drawback of such approach is the dependence on extensive sensor installation for localization and state estimation (Cai et al, 2021). Another approach is to use a modular pipeline (Kabzan et al, 2019;Strobel et al, 2020;Francis et al, 2022;Tatiya et al, 2022;Betz et al, 2022), starting from perception on raw sensory inputs, to localization and objectdetection, and finally to planning and control. While this approach is most commonly used in practice, disadvantages of the approach include over-complexity and error propagation (Yurtsever et al, 2020;Francis et al, 2022).…”

Section: Related Workmentioning

confidence: 99%

“…To add to the challenge, the vision encoder is subject to stringent requirements on inference time, due to the nature of the autonomous racing task. High latency can adversely impact agent performance, where, as discussed in prior art (Strobel et al, 2020), perception stacks in autonomous race-cars account for nearly 60% of total latency. Moreover, even though we want to maximise performance on the tracks that the agent directly interacts with, we do not want to overfit to the training tracks; the visual encoder must generalise to unseen contexts.…”

Section: Insights and Future Directionsmentioning

confidence: 99%

Learn-to-Race Challenge 2022: Benchmarking Safe Learning and Cross-domain Generalisation in Autonomous Racing

Francis¹,

Chen²,

Ganju³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present the results of our autonomous racing virtual challenge, based on the newly-released Learn-to-Race (L2R) simulation framework, which seeks to encourage interdisciplinary research in autonomous driving and to help advance the state of the art on a realistic benchmark. Analogous to racing being used to test cutting-edge vehicles, we envision autonomous racing to serve as a particularly challenging proving ground for autonomous agents as: (i) they need to make sub-second, safety-critical decisions in a complex, fast-changing environment; and (ii) both perception and control must be robust to distribution shifts, novel road features, and unseen obstacles. Thus, the main goal of the challenge is to evaluate the joint safety, performance, and generalisation capabilities of reinforcement learning agents on multi-modal perception, through a two-stage process. In the first stage of the challenge, we evaluate an autonomous agent's ability to drive as fast as possible, while adhering to safety constraints. In the second stage, we additionally require the agent to adapt to an unseen racetrack through safe exploration. In this paper, we describe the new L2R Task 2.0 benchmark, with refined metrics and baseline approaches. We also provide an overview of deployment, evaluation, and rankings for the inaugural instance of the L2R Autonomous Racing Virtual Challenge (supported by Carnegie Mellon University, Arrival Ltd., AICrowd, Amazon Web Services, and Honda Research), which officially used the new L2R Task 2.0 benchmark and received over 20,100 views, 437 active participants, 46 teams, and 733 model submissions-from 88+ unique institutions, in 58+ different countries. Finally, we release leaderboard results from the challenge and provide description of the two top-ranking approaches in cross-domain model transfer, across multiple sensor configurations and simulated races.

show abstract

Accurate, Low-Latency Visual Perception for Autonomous Racing: Challenges, Mechanisms, and Practical Solutions

Cited by 13 publications

References 21 publications

Accurate Mapping and Planning for Autonomous Racing

Accurate Mapping and Planning for Autonomous Racing

TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge

Learn-to-Race Challenge 2022: Benchmarking Safe Learning and Cross-domain Generalisation in Autonomous Racing

Contact Info

Product

Resources

About