Multi-granularity scenarios understanding network for trajectory prediction

Abstract

Understanding agents’ motion behaviors under complex scenes is crucial for intelligent autonomous moving systems (like delivery robots and self-driving cars). It is challenging duo to the inherent uncertain of future trajectories and the large variation in the scene layout. However, most recent approaches ignored or underutilized the scenario information. In this work, a Multi-Granularity Scenarios Understanding framework, MGSU, is proposed to explore the scene layout from different granularity. MGSU can be divided into three modules: (1) A coarse-grained fusion module uses the cross-attention to fuse the observed trajectory with the semantic information of the scene. (2) The inverse reinforcement learning module generates optimal path strategy through grid-based policy sampling and outputs multiple scene paths. (3) The fine-grained fusion module integrates the observed trajectory with the scene paths to generate multiple future trajectories. To fully explore the scene information and improve the efficiency, we present a novel scene-fusion Transformer, whose encoder is used to extract scene features and the decoder is used to fuse scene and trajectory features to generate future trajectories. Compared with the current state-of-the-art methods, our method decreases the ADE errors by 4.3% and 3.3% by gradually integrating different granularity of scene information on SDD and NuScenes, respectively. The visualized trajectories demonstrate that our method can accurately predict future trajectories after fusing scene information.

Human Motion Prediction via Dual-Attention and Multi-Granularity Temporal Convolutional Networks

“…By adjusting the convolutional size, CNN-based deep learning models can quickly gather feature information at various granularities, enabling more accurate decision making by combining and evaluating data from various scales. Recent achievements in the field of computer vision have fully exploited multi-granularity information fusion based on CNNs [ 39 ].…”

Underwater Source Counting with Local-Confidence-Level-Enhanced Density Clustering

“…Therefore, there are l ω l m time-frequency points in the rectangular area Ω ω,m . The local confidence Γ(ω, m) of this time-frequency point (ω, m) is estimated by performing a principal component analysis (PCA) [30] on the snapshot vector Y(Ω ω,m ) in the region Ω ω,m , which reflects the strength of the dominant signal at the time-frequency point (ω, m). It is obvious that the local confidence level is proportional to the ratio of the dominant signal.…”

A federated pedestrian trajectory prediction model with data privacy protection

“…We pre-process the trajectory data by extracting 20 consecutive frames to form a sample, in which the first eight frames are input, and the last 12 frames are the ground truth. Therefore, the observed and predicted horizons are 8 (3.2 s) and 12 (4.8 s) time steps [49], respectively.…”