A nonconvex low-rank tensor completion model for spatiotemporal traffic data imputation

Abstract

Missing value problem in spatiotemporal traffic data has long been a challenging topic, in particular for largescale and high-dimensional data with complex missing mechanisms and diverse degrees of missingness. Recent studies based on tensor nuclear norm have demonstrated the superiority of tensor learning in imputation tasks by effectively characterizing the complex correlations/dependencies in spatiotemporal data. However, despite the promising results, these approaches do not scale well to large tensors. In this paper, we focus on addressing the missing data imputation problem for large-scale spatiotemporal traffic data. To achieve both high accuracy and efficiency, we develop a scalable autoregressive tensor learning model-Low-Tubal-Rank Autoregressive Tensor Completion (LATC-Tubal)-based on the existing framework of Low-Rank Autoregressive Tensor Completion (LATC by Chen and Sun (2020)), which is well-suited for spatiotemporal traffic data that characterized by multidimensional structure of location× time of day × day. In particular, the proposed LATC-Tubal model involves a scalable tensor nuclear norm minimization scheme by integrating linear unitary transformation. Therefore, the tensor nuclear norm minimization can be solved by singular value thresholding on the transformed matrix of each day while the day-today correlation can be effectively preserved by the unitary transform matrix. Before setting up the experiment, we consider two large-scale 5-minute traffic speed data sets collected by the California PeMS system with 11160 sensors: 1) PeMS-4W covers the data over 4 weeks (i.e., 288 × 28 time points), and 2) PeMS-8W covers the data over 8 weeks (i.e., 288 × 56 time points). We compare LATC-Tubal with state-of-the-art baseline models, and find that LATC-Tubal can achieve competitively accuracy with a significantly lower computational cost. In addition, the LATC-Tubal will also benefit other tasks in modeling large-scale spatiotemporal traffic data, such as network-level traffic forecasting.

A Higher-Order Motif-Based Spatiotemporal Graph Imputation Approach for Transportation Networks

“…(i) LRTC-TNN [14] is a low-rank tensor completion framework with truncated nuclear norm, which extracts spatiotemporal features from traffic data (ii) SSIM [15] is a sequence-to-sequence imputation model, which is designed to impute missing data by utilizing the LSTM from both the past and future time indexes (iii) MGIA: our proposed approach, which incorporates the motif-based graph aggregation method with the multitime dimension fusion method based on Bi-LSTM, to impute missing data (iv) GIA is a comparison for MGIA, which incorporates the graph aggregation method with the multitime dimension fusion method based on Bi-LSTM but does not include motif-based application 4.2.2. Evaluation Metrics.…”

“…Li et al developed a combined deep neural model, which extracted spatio-temporal features to estimate missing values [13]. To characterize the hidden patterns in spatiotemporal traffic data, Chen et al incorporated a low-rank tensor completion (LRTC) framework with the truncated nuclear norm (TNN) and obtained a better solution for data imputation [14]. Considering the case of continuous data missing, Zhang et al utilized the temporal neighbouring values of a given period and employed the long short-term memory network (LSTM) to recover missing data [15].…”

AI4Water v1.0: An open source python package for modeling hydrological time series using data-driven methods

“…Several other libraries have been developed that have dedicated algorithms for imputing missing time series data. These include fancyimpute (Rubinsteyn and Feldman, 2016) and transdim (Chen et al, 2020b). The fancyimpute library provides several state-of-the-art algorithms such as SoftImpute (Mazumder et al, 2010), IterativeSVD (Troyanskaya et al, 2001), MatrixFactorization, NuclearNormMinimization (Candès and Recht, 2009), and Biscaler (Hastie et al, 2015).…”

Expressway Traffic Flow Missing Data Repair Method Based on Coupled Matrix-Tensor Factorizations

“…e importance of auxiliary traffic flow data in data repair is explained as follows: Firstly, different weather conditions may affect the operational behaviours of vehicles, for example, in rainy weather, due to the slippery road surface, which will directly affect the speed of vehicles, thus leading to similar running state of vehicles in similar weather conditions. Secondly, the analysis of the running speed of vehicles in different lanes of the same section shows that the vehicles using the middle lane will travel at different speeds compared to the ones in the edge lanes [32,33]. e difference in the information added for the traffic flow data repair helps to improve the restoration precision of the traffic flow data.…”