Probabilistic Future Prediction for Video Scene Understanding

Hu, Anthony; Cotter, Fergal; Mohan, Nikhil; Gurău, Corina; Kendall, Alex

doi:10.1007/978-3-030-58517-4_45

Cited by 50 publications

(42 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Authors A. Hu et al [102] presented a deep learning probabilistic model for the autonomous vehicle's video scene understanding of real-world urban scenes. This model learns features from the spatio-temporal convolutional network to predict future scene representation jointly by encoding the future state into a low-dimensional future distribution.…”

Section: ) Hybrid Approachmentioning

confidence: 99%

Video Processing Using Deep Learning Techniques: A Systematic Literature Review

et al. 2021

View full text Add to dashboard Cite

Studies show lots of advanced research on various data types such as image, speech, and text using deep learning techniques, but nowadays, research on video processing is also an emerging field of computer vision. Several surveys are present on video processing using computer vision deep learning techniques, targeting specific functionality such as anomaly detection, crowd analysis, activity monitoring, etc. However, a combined study is still unexplored. This paper aims to present a Systematic Literature Review (SLR) on video processing using deep learning to investigate the applications, functionalities, techniques, datasets, issues, and challenges by formulating the relevant research questions (RQs). This systematic mapping includes 93 research articles from reputed databases published between 2011 and 2020. We categorize the deep learning technique for video processing as CNN, DNN, and RNN based. We observe the significant advancements in video processing between 2017 and 2020, primarily due to the advent of AlexNet, ResNet, and LSTM based deep learning techniques. The prominent fields of video processing research are observed as human action recognition, crowd anomaly detection, and behavior analysis. This SLR is a helpful guide for the researchers to explore the recent literature, available datasets, and existing deep learning techniques for video processing.

show abstract

Section: ) Hybrid Approachmentioning

confidence: 99%

Video Processing Using Deep Learning Techniques: A Systematic Literature Review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…However, Bayesian methods are known for slow inference and poor real-time performance. Multi-modality can also be expressed within a conditional variational framework [ 4 ], by modelling interaction between the static scene, moving objects and multiple moving objects. However, the reported performance suggests that the task is far from being solved.…”

Section: Related Workmentioning

confidence: 99%

“…However, the ability to predict the future is an even more important attribute of intelligent behavior [ 16 , 21 , 23 – 25 ]. It is intuitively clear that critical real-time systems such as autonomous driving controllers could immensely benefit from the ability to predict the future by considering the past [ 4 , 17 , 27 ]. Such systems could make much better decisions than their counterparts which are able to perceive only the current moment.…”

Section: Introductionmentioning

confidence: 99%

“…Sometimes we can deduce what could be in that new space by observing recent past, and sometimes we simply can’t know. In both cases, we would like our model to produce a distribution over all possible outcomes in a stochastic environment [ 1 , 4 , 9 , 17 ]. We will address this goal by converting the basic regression model into a conditional generative model based on adversarial learning [ 18 ] and moment reconstruction losses [ 12 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Semantic Forecasting Based on Conditional Generation of Future Features

Fugošić

Šarić

Šegvić

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper considers semantic forecasting in road-driving scenes. Most existing approaches address this problem as deterministic regression of future features or future predictions given observed frames. However, such approaches ignore the fact that future can not always be guessed with certainty. For example, when a car is about to turn around a corner, the road which is currently occluded by buildings may turn out to be either free to drive, or occupied by people, other vehicles or roadworks. When a deterministic model confronts such situation, its best guess is to forecast the most likely outcome. However, this is not acceptable since it defeats the purpose of forecasting to improve security. It also throws away valuable training data, since a deterministic model is unable to learn any deviation from the norm. We address this problem by providing more freedom to the model through allowing it to forecast different futures. We propose to formulate multimodal forecasting as sampling of a multimodal generative model conditioned on the observed frames. Experiments on the Cityscapes dataset reveal that our multimodal model outperforms its deterministic counterpart in short-term forecasting while performing slightly worse in the mid-term case. Electronic supplementary material The online version of this chapter (10.1007/978-3-030-71278-5_34) contains supplementary material, which is available to authorized users.

show abstract

“…Furthermore, the trajectory prediction is now becoming a heated research focus in the field of computer vision using the image process algorithms. However, it heavily depends on the data set and is short of interpretability (Hu et al, 2020). Compared with the physics-based method, the prediction accuracy of the maneuver-based method in the short term is lower because it ignores the low-level characteristics of the vehicle in most cases such as vehicle lateral and longitudinal accelerations.…”

Section: Declaration Of Competing Interestmentioning

confidence: 99%

Probabilistic vehicle trajectory prediction via driver characteristic and intention estimation model under uncertainty

Liu

Xiong

Wang

et al. 2020

View full text Add to dashboard Cite

Purpose For autonomous vehicles, trajectory prediction of surrounding vehicles is beneficial to improving the situational awareness of dynamic and stochastic traffic environments, which is a crucial and indispensable element to realize highly automated driving. Design/methodology/approach In this paper, the overall framework consists of two parts: first, a novel driver characteristic and intention estimation (DCIE) model is built to indicate the higher-level information of the vehicle using its low-level motion variables; then, according to the estimation results of the DCIE model, a classified Gaussian process model is established for probabilistic vehicle trajectory prediction under different motion patterns. Findings The whole method is later applied and analyzed in the highway lane-change scenarios with the parameters of models learned from the public naturalistic driving data set. Compared with other traditional methods, the performance of this proposed approach is proved superior, demonstrated by the higher accuracy in the long prediction horizon and a more reasonable description of uncertainty. Originality/value This hierarchical approach is proposed to make trajectory prediction accurately both in the short term and long term, which can also deal with the uncertainties caused by the perception system or indeterminate vehicle behaviors.

show abstract

Probabilistic Future Prediction for Video Scene Understanding

Cited by 50 publications

References 27 publications

Video Processing Using Deep Learning Techniques: A Systematic Literature Review

Video Processing Using Deep Learning Techniques: A Systematic Literature Review

Multimodal Semantic Forecasting Based on Conditional Generation of Future Features

Probabilistic vehicle trajectory prediction via driver characteristic and intention estimation model under uncertainty

Contact Info

Product

Resources

About