Learning Dynamic and Hierarchical Traffic Spatiotemporal Features With Transformer

Yan, Haoyang; Ma, Xiaolei; Pu, Ziyuan

doi:10.1109/tits.2021.3102983

Cited by 129 publications

(45 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiself-Attention Machine Module. Te attention mechanism was originally used to solve the problem of longterm sequence dependence in machine translation, in which the performance of machine translation decreased signifcantly with the increase in the sentence length [16]. It has been widely used in the processing of various time series data.…”

Section: Methodsmentioning

confidence: 99%

“…In such cases, deep graph neural networks (GNNs) are superior to extracting features from transportation networks. Research studies mainly focus on the graph convolution network [16,[38][39][40] and spectral convolution network [41]. Te graph convolution neural network builds the trafc graph based on the physical network topology and defnes a graph convolution neural network to capture spatial features.…”

Section: Advances In Civil Engineeringmentioning

confidence: 99%

“…Tus, many scholars put forward a hybrid model that adds an attention mechanism to the model of the CNN or RNN class [12][13][14][15][16], which can efectively mitigate above problems by the ability on extraction critical information [17]. Terefore, a potential self-attention-based hybrid model is proposed in this paper.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Effective Self-Attention-Based Hybrid Model for Short-Term Traffic Flow Prediction

Wang

Yang

2023

Advances in Civil Engineering

View full text Add to dashboard Cite

Vehicle exhaust is one of the main sources of carbon emissions. The short-term traffic flow prediction plays an important role in alleviating traffic congestion, optimizing the travel structure, and reducing traffic carbon emissions. The current advanced models of short-term traffic flow prediction are evaluated in this work, especially their inadequacies. To improve the prediction accuracy and ensure fine traffic management, an effective self-attention-based hybrid model is proposed to predict the short-term traffic flow. The proposed model includes an encoder-decoder neural network module and a self-attention mechanism module. The self-attention mechanism module is applied as a feature extraction unit in this hybrid model to enhance the ability of key information capture and to settle the problem on key information disappearing due to the increasing sequence length in traditional models. The dataset of the Guangdong freeway toll station is used for the experimental testing. Compared with several baseline models, the proposed model is more suitable for real-time prediction and can provide highly accurate results. Also, a better interpretability is presented in this proposed model. The experimental results showed that MAE, RMSE, and MAPE of the proposed model are 3.01, 4.38, and 12.99%, respectively. Our new hybrid model gives a higher accuracy than the support vector regression (SVR) model, LSTM neural network-attention (LSTM-attention) model, and temporal convolutional network (TCN) model. It shows that the proposed model in this work is favorable to the short-term traffic flow prediction.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Advances In Civil Engineeringmentioning

confidence: 99%

See 1 more Smart Citation

An Effective Self-Attention-Based Hybrid Model for Short-Term Traffic Flow Prediction

Wang

Yang

2023

Advances in Civil Engineering

View full text Add to dashboard Cite

show abstract

“…Nicolas et al [3] proposed DETR, which applied a transformer to object detection task. Yan et al use a transformer to predict long-term traffic flow [33]. Cai et al [34] use a transformer to capture the spatial dependency for continuity and periodicity time series.…”

Section: Transformers Structurementioning

confidence: 99%

DetectFormer: Category-Assisted Transformer for Traffic Scene Object Detection

Liang

Bao

Pan

et al. 2022

Sensors

View full text Add to dashboard Cite

Object detection plays a vital role in autonomous driving systems, and the accurate detection of surrounding objects can ensure the safe driving of vehicles. This paper proposes a category-assisted transformer object detector called DetectFormer for autonomous driving. The proposed object detector can achieve better accuracy compared with the baseline. Specifically, ClassDecoder is assisted by proposal categories and global information from the Global Extract Encoder (GEE) to improve the category sensitivity and detection performance. This fits the distribution of object categories in specific scene backgrounds and the connection between objects and the image context. Data augmentation is used to improve robustness and attention mechanism added in backbone network to extract channel-wise spatial features and direction information. The results obtained by benchmark experiment reveal that the proposed method can achieve higher real-time detection performance in traffic scenes compared with RetinaNet and FCOS. The proposed method achieved a detection performance of 97.6% and 91.4% in AP50 and AP75 on the BCTSDB dataset, respectively.

show abstract

“…Yan et al [45] improved the transformer structure by introducing a global decoder and a global-local decoder respectively. The multi-head attention is used to extract nonlocal features, and the masked multi-head attention is focused on extracting local features.…”

Section: Differences Between Sdscnn and Existing Workmentioning

confidence: 99%

SDSCNN: A Hybrid Model Integrating Static and Dynamic Spatial Correlation Neural Network for Traffic Prediction

et al. 2022

View full text Add to dashboard Cite

Traffic flow prediction is of great significance for traffic control, and it has been challenging for capturing the complex spatial-temporal correlation. However, most existing prediction methods only consider the spatial adjacency of the nodes (i.e., static spatial correlation), lacking sufficient analysis of nonstationary traffic conditions (i.e., dynamic spatial correlation). The combination of static spatial correlation and dynamic spatial correlation enables the model to comprehensively analyze the feature of traffic flow at each moment and improve the mining capability. To address this problem, we use the multi-head selfattention mechanism to establish a hybrid model integrating static and dynamic spatial correlation neural network (SDSCNN) for traffic flow prediction. Specifically, we first construct static adjacency matrix and dynamic adjacency matrix according to different methods. These two matrices are simultaneously input into Graph Attention Network for analysis. The two outputs are integrated by the sum operation. Then the fused static and dynamic spatial features are fed into the multi-head self-attention layer to analyze the temporal correlation. Also, multi-layer SDSCNNs are stacked to further analyze the dynamic correlations between road sections, as well as to improve the model's multi-step prediction capability. Finally, Multi-layer Perceptron is used to output the prediction results. Extensive experiments are conducted using the datasets PEMS04, PEMS08, and METR-LA. And the results demonstrate that our model shows a good prediction performance.

show abstract

Learning Dynamic and Hierarchical Traffic Spatiotemporal Features With Transformer

Cited by 129 publications

References 33 publications

An Effective Self-Attention-Based Hybrid Model for Short-Term Traffic Flow Prediction

An Effective Self-Attention-Based Hybrid Model for Short-Term Traffic Flow Prediction

DetectFormer: Category-Assisted Transformer for Traffic Scene Object Detection

SDSCNN: A Hybrid Model Integrating Static and Dynamic Spatial Correlation Neural Network for Traffic Prediction

Contact Info

Product

Resources

About