Towards Streaming Perception

Li, Mengtian; Wang, Yu-Xiong; Ramanan, Deva

doi:10.1007/978-3-030-58536-5_28

Cited by 87 publications

(83 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Being deployable in an incremental fashion is a must for autonomous driving applications; this is a quality that other tubelet-based online action detection methods [81], [82], [87] fail to exhibit, as they can only be deployed in a sliding window fashion. Interestingly, the latest work on streaming object detection [104] proposes an approach that integrates latency and accuracy into a single metric for real-time online perception, termed 'streaming accuracy'. We will consider adopting this metric in the future evolution of ROAD.…”

Section: Results On the Various Tasksmentioning

confidence: 99%

ROAD: The Road Event Awareness Dataset for Autonomous Driving

Singh

Akrigg

Maio

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Humans drive in a holistic fashion which entails, in particular, understanding dynamic road events and their evolution. Injecting these capabilities in autonomous vehicles can thus take situational awareness and decision making closer to human-level performance. To this purpose, we introduce the ROad event Awareness Dataset (ROAD) for Autonomous Driving, to our knowledge the first of its kind. ROAD is designed to test an autonomous vehicle's ability to detect road events, defined as triplets composed by an active agent, the action(s) it performs and the corresponding scene locations. ROAD comprises videos originally from the Oxford RobotCar Dataset, annotated with bounding boxes showing the location in the image plane of each road event. We benchmark various detection tasks, proposing as a baseline a new incremental algorithm for online road event awareness termed 3D-RetinaNet. We also report the performance on the ROAD tasks of Slowfast and YOLOv5 detectors, as well as that of the winners of the ICCV2021 ROAD challenge, which highlight the challenges faced by situation awareness in autonomous driving. ROAD is designed to allow scholars to investigate exciting tasks such as complex (road) activity detection, future event anticipation and continual learning. The dataset is available at https://github.com/gurkirt/road-dataset; the baseline can be found at https://github.com/gurkirt/3D-RetinaNet.

show abstract

Section: Results On the Various Tasksmentioning

confidence: 99%

ROAD: The Road Event Awareness Dataset for Autonomous Driving

Singh

Akrigg

Maio

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Multi-dataset Fine-tuning Stage. Due to the limited data provided in Argoverse-HD [7] train set, we utilize several additional datasets to enhance the model capacity. Specifically, though there are 39384 images in train set, which mainly comes from only 65 video sequences.…”

Section: Training Configurationmentioning

confidence: 99%

Workshop on Autonomous Driving at CVPR 2021: Technical Report for Streaming Perception Challenge

Zhang¹,

Lin²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this report, we introduce our real-time 2D object detection system for realistic autonomous driving scenario. Our detector is built on a new designed YOLO model, called YOLOX. On the Argoverse-HD dataset, our system achieves 41.0 streaming AP, which surpassed the second place by 7.8/6.1 on detection-only track/fully track, respectively. Moreover, equipped with TensorRT, our model achieves the 30FPS inference speed with a high-resolution input size (e.g., 1440×2304). Code and models will be available at https://github.com/Megvii-BaseDetection/YOLOX * equal contribution. This work is done when Songyang Zhang, Lin Song and Zheng Ge are research interns at Megvii Technology.

show abstract

“…However, they generally do not evaluate models according to runtime, hence assuming computational resources not to be finite. Some works have recently considered a "streaming" scenario in which algorithms should be evaluated considering the time in which the predictions are available depending on model runtime [25,26].…”

Section: Related Workmentioning

confidence: 99%

“…We build on works on efficient computer vision [7,19], online video processing [5] and streaming perception [26]. However, differently from these works, we study the streaming scenario within the task of egocentric action anticipation, in which the timeliness of predictions is fundamental to ensure their practical utility.…”

Section: Related Workmentioning

confidence: 99%

“…Nevertheless, previous works have not considered the computational time required to process the input video, which may lead the predictions to be late. Actually, little attention has been paid to the effect of model runtime on performance in scenarios in which predictions need to be used in real time in general [26], and egocentric action anticipation approaches, in particular, have been evaluated assuming a negligible runtime [4,12,41,29,51,53,34]. We argue that this assumption can lead to unfair and overly optimistic evaluations and propose that egocentric action anticipation should be evaluated considering a "streaming" scenario.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation