Learning Spatiotemporal Latent Factors of Traffic via Regularized Tensor Factorization: Imputing Missing Values and Forecasting

Abstract

Intelligent transportation systems are a key component in smart cities, and the estimation and prediction of the spatiotemporal traffic state is critical to capture the dynamics of traffic congestion, i.e., its generation, propagation and mitigation, in order to increase operational efficiency and improve livability within smart cities. And while spatiotemporal data related to traffic is becoming common place due to the wide availability of cheap sensors and the rapid deployment of IoT platforms, the data still suffer some challenges related to sparsity, incompleteness, and noise which makes the traffic analytics difficult. In this paper, we investigate the problem of missing data or noisy information in the context of real-time monitoring and forecasting of traffic congestion for road networks in a city. The road network is represented as a directed graph in which nodes are junctions (intersections) and edges are road segments. We assume that the city has deployed high-fidelity sensors for speed reading in a subset of edges; and the objective is to infer the speed readings for the remaining edges in the network; and to estimate the missing values in the segments for which sensors have stopped generating data due to technical problems (e.g., battery, network, etc.). We propose a tensor representation for the series of road network snapshots, and develop a regularized factorization method to estimate the missing values, while learning the latent factors of the network. The regularizer, which incorporates spatial properties of the road network, improves the quality of the results. The learned factors, with a graph-based temporal dependency, are then used in an autoregressive algorithm to predict the future state of the road network with a large horizon. Extensive numerical experiments with real traffic data from the cities of Doha (Qatar) and Aarhus (Denmark) demonstrate that the proposed approach is appropriate for imputing the missing data and predicting the traffic state. It is accurate and efficient and can easily be applied to other traffic datasets.

Missing Traffic Data Imputation for Artificial Intelligence in Intelligent Transportation Systems: Review of Methods, Limitations, and Challenges

“…Information regarding the base method used, the referenced papers, the type of road network, the method of data acquisition (i.e., Public, private, or manually collected dataset), and the type of missing data tested was summarised in TABLE 1. Tensor Decomposition [69], [70], [71], [72], [18], [73], [74], [75], [19], [76], [77], [78],…”

“…Even papers that focus on traffic forecasts, such as [68], make use of tensor decomposition to deal with their missing data before moving on to their proposed model. Papers such as [69], [70], [71], [72], [18], [73], [74], [75], [19], [76], [77], [78] and [79] are some of the recent state-of-the-art missing data imputation methods that have been proposed in the past three years that have utilized tensor factorization as a core part of their model. These tensor-based models performed well due to their being able to extract latent features from a traffic dataset and, through decomposition and completion, can fill in the missing blanks in an accurate manner.…”

“…These tensor-based models performed well due to their being able to extract latent features from a traffic dataset and, through decomposition and completion, can fill in the missing blanks in an accurate manner. Via Bayesian Statistics ( [69], [71], [74]), extending or modifying the existing tensor factorization methods ( [70], [72], [18], [75], [19]), and even adding an additional preprocessing method ( [76], [79]), the base tensor factorization method has shown a significant improvement in the field of missing traffic data imputation. This can also be seen as a majority of these models have been tested for robustness in imputing missing traffic data of rates ranging from 1% to 90% while still retaining a high level of accuracy when compared to their respective benchmarks.…”

Traffic Flow Prediction Based on Hybrid Deep Learning Models Considering Missing Data and Multiple Factors

“…Tensors provide a simple and effective approach to represent spatio-temporal traffic flows. Therefore, several scholars have adopted a tensor-based approach to complement traffic flow interpolation and prediction [15][16][17]. The graph Laplacian method offers an efficient approach for extracting spatio-temporal information, which can be combined with LSTM to achieve accurate predictions in scenarios with missing data [18].…”

Integrated Node Infrastructure for Future Smart City Sensing and Response

“…Firstly, the different interface protocols cannot be unified, accessed, and managed in the legacy system. There is an illustration of the in situ site access: The Low-Power Wide-Area Network (LpWA) sensors upload to the Sensor Observation Service (SOS) directly [21][22][23]. The sensors working with the Low-Power Personal Area Network (LoWAN) need to install network infrastructures, such as radio transceivers, gateways, and Remote Terminal Units (RTU) to suit the protocol for data acquisition and upload [24].…”