A survey of accelerator architectures for 3D convolution neural networks

Mittal, Sparsh; Vibhu,

doi:10.1016/j.sysarc.2021.102041

Cited by 38 publications

(19 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…One disadvantage is that this method fails to utilise the spatial distribution of Doppler velocities. Alternatively, 3D convolution can be used to extract features from all three dimensions in a 3D tensor, but requires huge computation and memory overheads [161]. RODNet [47] samples chirp sequences, as described above, to reduce input dimensionality.…”

Section: Pre-cfar Detectormentioning

confidence: 99%

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

Zhou

Liu

Zhao

et al. 2022

Sensors

View full text Add to dashboard Cite

With recent developments, the performance of automotive radar has improved significantly. The next generation of 4D radar can achieve imaging capability in the form of high-resolution point clouds. In this context, we believe that the era of deep learning for radar perception has arrived. However, studies on radar deep learning are spread across different tasks, and a holistic overview is lacking. This review paper attempts to provide a big picture of the deep radar perception stack, including signal processing, datasets, labelling, data augmentation, and downstream tasks such as depth and velocity estimation, object detection, and sensor fusion. For these tasks, we focus on explaining how the network structure is adapted to radar domain knowledge. In particular, we summarise three overlooked challenges in deep radar perception, including multi-path effects, uncertainty problems, and adverse weather effects, and present some attempts to solve them.

show abstract

Section: Pre-cfar Detectormentioning

confidence: 99%

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

Zhou

Liu

Zhao

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…CNNs have found numerous applications in video processing [32,49,59]. These include LSTM-based networks that perform per-frame encoding [44,50,59] and the extension of 2D convolutions to the temporal dimension, e.g., 3D CNNs such as C3D [48], R2D [43] and R(2+1)D [49].…”

Section: Deep Learning For Video Understandingmentioning

confidence: 99%

GraphVid: It Only Takes a Few Nodes to Understand a Video

Kosman¹,

Castro²

2022

Preprint

View full text Add to dashboard Cite

We propose a concise representation of videos that encode perceptually meaningful features into graphs. With this representation, we aim to leverage the large amount of redundancies in videos and save computations. First, we construct superpixel-based graph representations of videos by considering superpixels as graph nodes and create spatial and temporal connections between adjacent superpixels. Then, we leverage Graph Convolutional Networks to process this representation and predict the desired output. As a result, we are able to train models with much fewer parameters, which translates into short training periods and a reduction in computation resource requirements. A comprehensive experimental study on the publicly available datasets Kinetics-400 and Charades shows that the proposed method is highly cost-effective and uses limited commodity hardware during training and inference. It reduces the computational requirements 10-fold while achieving results that are comparable to state-ofthe-art methods. We believe that the proposed approach is a promising direction that could open the door to solving video understanding more efficiently and enable more resource limited users to thrive in this research field.

show abstract

“…They demonstrated the success of their approach in relatively simple scenarios while being simple to train and deploy. Du et al [8] proposed more sophisticated models to perform steering angle prediction using 3D-CNN [25] and Long Short-Term Memory (LSTM) modules [12]. State-of-the-art performance was achieved in the Udacity challenge [41] Another line of works incorporates multi-task learning [31], side-task learning, or auxiliary task learning, in order to enhance the performance of a model on its main task.…”

Section: Related Workmentioning

confidence: 99%

“…Consequently, most contributions in this field focus on extending image-based algorithms with the temporal dimension. Examples are SIFT-3D [35], HOG3D [18], Action-Bank [32], and adaptions of convolutional neural networks [25,39,50]. Attempts to adapt the latter include LSTM-based network that firstly perform per-frame encoding [37,42,50].…”

Section: Introductionmentioning

confidence: 99%

Vision-Guided Forecasting -- Visual Context for Multi-Horizon Time Series Forecasting

Kosman¹,

Castro²

2021

Preprint

View full text Add to dashboard Cite

Autonomous driving gained huge traction in recent years, due to its potential to change the way we commute. Much effort has been put into trying to estimate the state of a vehicle. Meanwhile, learning to forecast the state of a vehicle ahead introduces new capabilities, such as predicting dangerous situations. Moreover, forecasting brings new supervision opportunities by learning to predict richer a context, expressed by multiple horizons. Intuitively, a video stream originated from a front facing camera is necessary because it encodes information about the upcoming road. Besides, historical traces of the vehicle's states gives more context. In this paper we tackle multi-horizon forecasting of vehicle states by fusing the two modalities. We design and experiment with 3 end-to-end architectures that exploit 3D convolutions for visual features extraction and 1D convolutions for features extraction from speed and steering angle traces. To demonstrate the effectiveness of our method, we perform extensive experiments on two publicly available real-world datasets, Comma2k19 and the Udacity challenge. We show that we are able to forecast a vehicle's state to various horizons, while outperforming the current state of the art results on the related task of driving state estimation. We examine the contribution of vision features, and find that a model fed with vision features achieves an error that is 56.6% and 66.9% of the error of a model that doesn't use those features, on the Udacity and Comma2k19 datasets respectively.

show abstract

A survey of accelerator architectures for 3D convolution neural networks

Cited by 38 publications

References 31 publications

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges

GraphVid: It Only Takes a Few Nodes to Understand a Video

Vision-Guided Forecasting -- Visual Context for Multi-Horizon Time Series Forecasting

Contact Info

Product

Resources

About