An improved GAN with transformers for pedestrian trajectory prediction models

Abstract

Predicting the future trajectories of multiple pedestrians in certain scenes is critical for autonomous moving platforms (like, self-driving cars and social robots). In this paper, we propose a novel Generative Adversarial Network model with Transformers, which simulates the pedestrian distribution to capture the uncertainty of the predicted paths and generate more reasonable future trajectories. The design of our method includes a generator and a discriminator. The generator mainly contains an encoder, a decoder, and a prediction module. Specifically, the encoder and the decoder comprise multihead convolutional selfattention to learn the sequence of historical movement, and the prediction module incorporates the Mish Feed-Forward Network to yield the predicted target. The discriminator takes both the predicted paths and ground truth as input, classifies them as socially acceptable or not. Experimental results show that the proposed method consistently boosts the performance of trajectory forecasting, and our framework surpasses several existing baselines by evaluating the results on various data sets. Code is available at https://github. com/lzz970818/Trajectory-Prediction.

Ten years of generative adversarial nets (GANs): a survey of the state-of-the-art

“…TransGAN [229] introduces a GAN architecture without convolutions by using Transformers in both G and D of the GAN, resulting in improved high-resolution image generation. Lv et al [230] presented an intersection of GANs and transformers to predict pedestrian paths. Although transformers and their variants have several advantages, they suffer from high computational (time and resource) complexity [231].…”

Deep sparse autoencoder integrated with three‐stage framework for glaucoma diagnosis

“…Hence, how to design suitable features is a difficult and challenging problem in traditional machine learning approaches. 23 Nowadays, deep feature learning techniques have been achieved excellent performance in many research fields, such as image classification, [24][25][26] medical image processing, 27 speech recognition, 28 anomaly detection, 29 pedestrian trajectory prediction, 30 gesture segmentation, 31 privacy protection, 32 image registration, 33 and pose transfer. 34 Convolutional Neural Network (CNN) can be regarded as one of the most typical deep feature learning techniques.…”

FC‐ACGAN‐based data augmentation for terahertz time‐domain spectral concealed hazardous materials identification

“…18 Lv et al proposed a novel GAN combined with a Transformer in the application of pedestrian trajectory prediction, which improved the performance by introducing the Transformer structure into the generator. 19 Ran et al proposed an end-to-end network RGAN based on the encoder-decoder structure to achieve cloud removal in optical remote sensing images, which realized a more realistic cloud removal effect by introducing a soft attention mechanism and adaptive padding convolution into the decoder. 20 To automatically detect malware on Android better, Chen et al proposed an adversarial learning-based attack method that can generate strong adversarial examples without human intervention to improve malware detector performance.…”

“…Zhang et al designed fully connected ACGAN (FC‐ACGAN) by adding convolution layers and fully connected (FC) layers 17 to the discriminator and generator, to solve the problem of the small data set on solar cell electroluminescence image and achieved higher accuracy 18 . Lv et al proposed a novel GAN combined with a Transformer in the application of pedestrian trajectory prediction, which improved the performance by introducing the Transformer structure into the generator 19 . Ran et al proposed an end‐to‐end network RGAN based on the encoder–decoder structure to achieve cloud removal in optical remote sensing images, which realized a more realistic cloud removal effect by introducing a soft attention mechanism and adaptive padding convolution into the decoder 20 .…”