Ensemble correlation-based low-rank matrix completion with applications to traffic data imputation

Chen, Xiaobo; Wei, Zhongjie; Li, Zuoyong; Liang, Jun; Cai, Yingfeng; Zhang, Bob

doi:10.1016/j.knosys.2017.06.010

Cited by 69 publications

(29 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A correlation-based low-rank matrix completion (LRMC) method was developed by Chen et al [12]. The method applies LRMC to estimate missing data then uses a weighted Pearson's correlation followed by K-nearest neighbour (k-NN) search to choose the most similar samples.…”

Section: Related Workmentioning

confidence: 99%

Multiple Imputation Ensembles (MIE) for Dealing with Missing Data

2020

View full text Add to dashboard Cite

Missing data is a significant issue in many real-world datasets, yet there are no robust methods for dealing with it appropriately. In this paper, we propose a robust approach to dealing with missing data in classification problems: Multiple Imputation Ensembles (MIE). Our method integrates two approaches: multiple imputation and ensemble methods and compares two types of ensembles: bagging and stacking. We also propose a robust experimental set-up using 20 benchmark datasets from the UCI machine learning repository. For each dataset, we introduce increasing amounts of data Missing Completely at Random. Firstly, we use a number of single/multiple imputation methods to recover the missing values and then ensemble a number of different classifiers built on the imputed data. We assess the quality of the imputation by using dissimilarity measures. We also evaluate the MIE performance by comparing classification accuracy on the complete and imputed data. Furthermore, we use the accuracy of simple imputation as a benchmark for comparison. We find that our proposed approach combining multiple imputation with ensemble techniques outperform others, particularly as missing data increases.

show abstract

Section: Related Workmentioning

confidence: 99%

Multiple Imputation Ensembles (MIE) for Dealing with Missing Data

2020

View full text Add to dashboard Cite

show abstract

“…As a consequence, the data matrix usually has low-rank property, implying the number of independent rows (and columns) is much smaller than the size of matrix. Under such a circumstance, low-rank matrix completion (LRMC) [ 18 , 19 ] can be used to recover MVs through rank (or its surrogate nuclear norm) minimization on the whole matrix. Many efficient optimization algorithms have been developed to solve the problem, e.g., SVT [ 18 , 44 ], FPCA [ 45 ], ADMM [ 46 ], etc.…”

Section: Related Workmentioning

confidence: 99%

“…As a result, there exist underlying structures hidden from the raw high-dimensional sensor data. Until now, many MVs imputation algorithms have been developed in literatures, including mean imputation (MI), K-nearest neighbor (KNN) [ 15 ], support vector machine (SVM) [ 16 ], singular value decomposition (SVD) [ 17 ], probability principal component analysis (PPCA) [ 2 ], low-rank matrix completion (LRMC) [ 18 , 19 ], etc.…”

Section: Introductionmentioning

confidence: 99%

Kernel Sparse Representation with Hybrid Regularization for On-Road Traffic Sensor Data Imputation

Chen

Cai

et al. 2018

Sensors

Self Cite

View full text Add to dashboard Cite

The problem of missing values (MVs) in traffic sensor data analysis is universal in current intelligent transportation systems because of various reasons, such as sensor malfunction, transmission failure, etc. Accurate imputation of MVs is the foundation of subsequent data analysis tasks since most analysis algorithms need complete data as input. In this work, a novel MVs imputation approach termed as kernel sparse representation with elastic net regularization (KSR-EN) is developed for reconstructing MVs to facilitate analysis with traffic sensor data. The idea is to represent each sample as a linear combination of other samples due to inherent spatiotemporal correlation, as well as periodicity of daily traffic flow. To discover few yet correlated samples and make full use of the valuable information, a combination of l1-norm and l2-norm is employed to penalize the combination coefficients. Moreover, the linear representation among samples is extended to nonlinear representation by mapping input data space into high-dimensional feature space, which further enhances the recovery performance of our proposed approach. An efficient iterative algorithm is developed for solving KSR-EN model. The proposed method is verified on both an artificially simulated dataset and a public road network traffic sensor data. The results demonstrate the effectiveness of the proposed approach in terms of MVs imputation.

show abstract

“…In 2016, Duan et al [21] proposed a deep learning method to impute the missing traffic data, and the empirical results show that the average imputation error of this method is below 10 veh/5 min, and the imputation performance is better than the historical average model, ARIMA model, and BP neural network model. More recently, Chen et al [22] proposed an ensemble correlation-based low-rank matrix completion method which achieved better imputation performance than competing methods (including temporal average imputation and the PPCA imputation). In this method, low-rank matrix is used to represent traffic data and ensemble KNN learning is used to explore the relationship between the missing data and the complete data.…”

Section: Introductionmentioning

confidence: 99%

An Imputation Method for Missing Traffic Data Based on FCM Optimized by PSO-SVR

Shang

Yang

Gao

et al. 2018

Journal of Advanced Transportation

View full text Add to dashboard Cite

Missing traffic data are inevitable due to detector failure or communication failure. Currently, most of imputation methods estimated the missing traffic values by using spatial-temporal information as much as possible. However, it ignores an important fact that spatial-temporal information of the traffic missing data is often incomplete and unavailable. Moreover, most of the existing methods are verified by traffic data from freeway, and their applicability to urban road data needs to be further verified. In this paper, a hybrid method for missing traffic data imputation is proposed using FCM optimized by a combination of PSO algorithm and SVR. In this method, FCM is the basic algorithm and the parameters of FCM are optimized. Firstly, the patterns of missing traffic data are analyzed and the representation of missing traffic data is given using matrix-based data structure. Then, traffic data from urban expressway and urban arterial road are used to analyze spatial-temporal correlation of the traffic data for the determination of the proposed method input. Finally, numerical experiment is designed from three perspectives to test the performance of the proposed method. The experimental results demonstrate that the novel method not only has high imputation precision, but also exhibits good robustness.

show abstract

Ensemble correlation-based low-rank matrix completion with applications to traffic data imputation

Cited by 69 publications

References 31 publications

Multiple Imputation Ensembles (MIE) for Dealing with Missing Data

Multiple Imputation Ensembles (MIE) for Dealing with Missing Data

Kernel Sparse Representation with Hybrid Regularization for On-Road Traffic Sensor Data Imputation

An Imputation Method for Missing Traffic Data Based on FCM Optimized by PSO-SVR

Contact Info

Product

Resources

About