IRLSOT: Inverse reinforcement learning for scene‐oriented trajectory prediction

Abstract

Forecasting pedestrians' future trajectory in unknown complex environments is essential to autonomous navigation in real‐world applications, for example, for self‐driving cars and collision warnings. However, modern observed trajectory‐based prediction methods may easily over‐fit to complex or rare scenes because they do not entirely understand the correlations between scenes and trajectories. To address the over‐fitting problem, an Inverse Reinforcement Learning for Scene‐oriented Trajectory Prediction (IRLSOT) is proposed in this work. The authors' method can be divided into three modules. First, the inverse reinforcement learning module generates the optimal policy by extracting features from scenes and pedestrians' observed trajectories. A lightweight ENet is used to extract features from scenes. Afterwards, the path sampling module introduces a Gumbel Softmax Trick (GST) to improve the accuracy of optimal policy sampling. Different paths are generated on the basis of the optimal policies. Finally, the information fusion module uses the proposed Scene Based Attention (SBA) to fuse the path and trajectory information, then outputs the predicted trajectories. Comparison results show that IRLSOT improves performance on Stanford Drone Database(SDD) by 5.9%$\%$. Furthermore, the authors' test IRLSOT on multi‐agent scenarios and the authors' own data sets, and results demonstrate that IRLSOT can enhance the generalization of trajectory prediction to rare or new scenes.

Multi-granularity scenarios understanding network for trajectory prediction

“…The size of FC (s fc ) is set to 128. In the IRL module, the dimension of the 2D grid (d grid ) is [25,25], the initial state is set to [12,12], and the size of the scene feature (s sf ) is set to 64. In the scene-fusion Transformer of the fine-grained fusion module, the embedding size of the model (e model ) is set to 512, the number of the layer (N l ) is set to 6, the number of heads of the multi-head attention (heads) is set to 8, and the dropout of the network is set to 0.01.…”

“…IRLSOT IRLSOT [25] proposes inverse reinforcement learning for scene-oriented trajectory prediction to better forecast pedestrians' future trajectories under rare or complex environments.…”

“…A lightweight semantic segmentation network, ESPNet [24], is introduced to extract the semantic features of the scene. To better utilize the scene information, an inverse reinforcement learning (IRL) approach [25] is introduced to generate the optimal path strategy based on semantic features. Concretely, the main contributions of this paper can be summarized as follows:…”

Forecasting Future Trajectories with an Improved Transformer Network

“…With the developments of machine learning [3][4][5], researchers proposed two kinds of prediction methods, including model-driven [6,7] and data-driven [8][9][10]. For the former, some researchers used the Markov chain and Kalman filter [6,7] to perform trajectory prediction.…”

Pedestrians crossing intention anticipation based on dual‐channel action recognition and hierarchical environmental context

“…Lee et al [13] presented a recurrent neural network based on inverse optimal control to forecast pedestrians' and vehicles' future trajectories simultaneously. Meanwhile, He et al [14] achieved transferable trajectory prediction based on inverse reinforcement learning.…”