Peeking Into the Future: Predicting Future Person Activities and Locations in Videos

Liang, Junwei; Jiang, Lu; Niebles, Juan Carlos; Hauptmann, Alexander G.; Li, Feifei

doi:10.1109/cvpr.2019.00587

Cited by 265 publications

(254 citation statements)

References 23 publications

Supporting

Mentioning

254

Contrasting

Order By: Relevance

“…The standard formulation of trajectory prediction problem in the literature [10], [12] is in the 2D image space. With observed trajectories of all moving agents in the scene, including persons and vehicles, the task is to predict the moving trajectories of all agents for the next period of time, say 10 seconds, in the near future.…”

Section: A Methods Overviewmentioning

confidence: 99%

“…Sadeghian et al [11] incorporated scene context to human trajectory prediction based on GAN (Generative Adversarial Network). Reference [12] extracted multiple visual features, including each person's body keypoints and the scene semantic map to predict human behavior and model interaction with the surrounding environment. Reference [3] proposed a Bayesian framework to predict unobserved paths from previously observed motions and to transfer learned motion patterns to new scenes.…”

Section: A Human Trajectory Predictionmentioning

confidence: 99%

See 1 more Smart Citation

Pedestrian Motion Trajectory Prediction With Stereo-Based 3D Deep Pose Estimation and Trajectory Learning

et al. 2020

View full text Add to dashboard Cite

Existing methods for pedestrian motion trajectory prediction are learning and predicting the trajectories in the 2D image space. In this work, we observe that it is much more efficient to learn and predict pedestrian trajectories in the 3D space since the human motion occurs in the 3D physical world and and their behavior patterns are better represented in the 3D space. To this end, we use a stereo camera system to detect and track the human pose with deep neural networks. During pose estimation, these twin deep neural networks satisfy the stereo consistence constraint. We adapt the existing SocialGAN method to perform pedestrian motion trajectory prediction from the 2D to the 3D space. Our extensive experimental results demonstrate that our proposed method significantly improves the pedestrian trajectory prediction performance, outperforming existing state-of-the-art methods.

show abstract

Section: A Methods Overviewmentioning

confidence: 99%

Section: A Human Trajectory Predictionmentioning

confidence: 99%

Pedestrian Motion Trajectory Prediction With Stereo-Based 3D Deep Pose Estimation and Trajectory Learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Datasets like Human 3.6M, Moving MNIST and RobotPush were used for experimentation [9]. For capturing the spatial and temporal features in natural video's ,proposed an Encoder-Decoder CNN and convolutional LSTM model [10]; Proposed a method Deep Voxel flow that consists of a Fully-Convolutional Encoder-Decoder architecture containing with three convolutional and deconvolutional layers [12]; for predicting the video frame used Gated Recurrent units which confluence noise during training phase [13]; This paper focuses on pixel level video prediction Manju D , Seetha M, Sammulal P Frame Prediction-Noise Removal using Denoising Autoencoders C Frame Prediction-Noise Removal using Denoising Autoencoders that uses hierarchical approach [14]; Predicts the Pedestrian's behavior, by extracting the person appearance and pose features [16]; Proposed an SDC module for learning motion vector of objects in the frame prediction and a kernel for synthesizing the pixels [17,31]; Proposed a Meta-Learning an unsupervised learning rule for training networks [18]; Focused on action prediction sequences, for this proposed global-local temporal action prediction model [24,23,33]. Intuitively, all the last few predicted frames from different models suffer from blurriness.…”

Section: Literature Surveymentioning

confidence: 99%

Frame Prediction- Noise Removal using Denoising Autoencoders

D*,

2019

IJITEE

View full text Add to dashboard Cite

In the current times, tasks like Object Detection, Object tracking, Gesture prediction, Video prediction in computer vision are being solved effectively with models of deep learning . Video frame prediction involves predicting the next few frames of a video given the previous frame or frames as input. Currently, the challenge in video frame prediction is that the predicted future frames are blurry. This paper focuses on the removal of noise from the predicted image using Denoising Autoencoders, solve the above-addressed issue. The proposed work, trains LSTM model which generates future frames by giving a sequence of input frames. The predicted output is given as an input to the Denoising Autoencoders which tries to remove the blurry predictions. Our approach is implemented on Moving MNIST Dataset. The result of our proposed method improved accuracy and is compared with the accuracy of Denoising Autoencoders, LSTM, and LSTM along with Denoising Autoencoders

show abstract

“…Conventional methods focus on using egocentric features, e.g., previous trajectory, for trajectory prediction, which lacks considering some surrounding information, i.e., multi-agents, in traffic environments. For example, some works in trajectory prediction mainly focus on road-agents in homogeneous environments [2,4,10,11,21,30,31,34,39], which only consists of a single type of road-agent in a scene. However, in real driving environment, it is necessary to differentiate the interactions among different types of road-agents such as the difference between pedestrians and bikes or bikes and trucks.…”

Section: Introductionmentioning

confidence: 99%

Trajectory Prediction in Heterogeneous Environment via Attended Ecology Embedding

Lai

Xia²,

Lin

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Trajectory prediction is a highly desirable feature for safe navigation or autonomous vehicle in complex traffic. In this paper, we consider the practical environment of predicting trajectory in the heterogeneous traffic ecology. The proposed method has various applications in trajectory prediction problems and also in applied fields beyond tracking. One challenge stands out of the trajectory prediction-heterogeneous environment. Particularly, many factors should be considered in the environments, i.e., multiple types of road-agents, social interactions and terrains. The information is complicated and large that may result in inaccurate trajectory prediction. We propose two social and visual enforced attention modules to circumvent the problem and a variant of an Info-GAN structure to predict the trajectory with multi-modal behaviors. Experimental results show that the proposed method significantly outperforms state-of-the-art methods in both heterogeneous and homogeneous real environments. CCS CONCEPTS • Computing methodologies → Computer vision; • Computer systems organization → Robotic autonomy.

show abstract

Peeking Into the Future: Predicting Future Person Activities and Locations in Videos

Cited by 265 publications

References 23 publications

Pedestrian Motion Trajectory Prediction With Stereo-Based 3D Deep Pose Estimation and Trajectory Learning

Pedestrian Motion Trajectory Prediction With Stereo-Based 3D Deep Pose Estimation and Trajectory Learning

Frame Prediction- Noise Removal using Denoising Autoencoders

Trajectory Prediction in Heterogeneous Environment via Attended Ecology Embedding

Contact Info

Product

Resources

About