On the imputation of missing data for road traffic forecasting: New insights and novel techniques

Laña, Ibai; Olabarrieta, Ignacio; Vélez, Manuel; Ser, Javier Del

doi:10.1016/j.trc.2018.02.021

Cited by 88 publications

(46 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This also suggests that the LI is comparatively sound, since data are not likely to fluctuate rapidly in a short time frame. However, when the missing data interval lengths increase, the imputation accuracy degrades severely [22].…”

Section: LImentioning

confidence: 99%

“…Several studies have demonstrated improved prediction performance via application of these methods to classification or prediction models [21,22]. In [21], the authors demonstrated a neural network model of breast cancer diagnosis with the use of three statistical methods and three missing data machine learning methods.…”

Section: Introductionmentioning

confidence: 99%

“…In [21], the authors demonstrated a neural network model of breast cancer diagnosis with the use of three statistical methods and three missing data machine learning methods. In [22], the authors demonstrated that the conventional road traffic congestion prediction performance can be improved via application of a missing data imputation method based on machine learning techniques.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Analysis and Impact Evaluation of Missing Data Imputation in Day-ahead PV Generation Forecasting

Kim

2019

Applied Sciences

View full text Add to dashboard Cite

Over the past decade, PV power plants have increasingly contributed to power generation. However, PV power generation widely varies due to environmental factors; thus, the accurate forecasting of PV generation becomes essential. Meanwhile, weather data for environmental factors include many missing values; for example, when we estimated the missing values in the precipitation data of the Korea Meteorological Agency, they amounted to ~16% from 2015–2016, and further, 19% of the weather data were missing for 2017. Such missing values deteriorate the PV power generation prediction performance, and they need to be eliminated by filling in other values. Here, we explore the impact of missing data imputation methods that can be used to replace these missing values. We apply four missing data imputation methods to the training data and test data of the prediction model based on support vector regression. When the k-nearest neighbors method is applied to the test data, the prediction performance yields results closest to those for the original data with no missing values, and the prediction model’s performance is stable even when the missing data rate increases. Therefore, we conclude that the most appropriate missing data imputation for application to PV forecasting is the KNN method.

show abstract

Section: LImentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Analysis and Impact Evaluation of Missing Data Imputation in Day-ahead PV Generation Forecasting

Kim

2019

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…In the research of traffic forecasting, the representative patent of Wynter et al shows usage of historical data as input to an exemplary missing data prediction algorithm [14] . Similarly, Laña et al present ML approaches to impute lacking traffic information, based on regression and classification models [17]. Although most of the complex imputation problems are searching for solutions in ML methodology, the dimension of ML opens structural and parametric optimisation problems, also addressed in [18,19].…”

Section: Introductionmentioning

confidence: 99%

Energetic Map Data Imputation: A Machine Learning Approach

et al. 2020

View full text Add to dashboard Cite

Despite a rapid increase of public interest for electric mobility, several factors still impede Battery Electric Vehicles’ (BEVs) acceptance. These factors include their limited range and inconvenient charging. For mitigating these limitations to users, certain BEV-specific services are required. Therefore, such services provide a reliable range prediction and routing, including charging-stop planning. The basis of these services is a precise and reliable Energy Demand (ED) prediction. For that matter, aggregated fleet-vehicle data combined with map-specific data (e.g., road slope) form an energetic map, which can serve for precise ED predictions. However, data coverage is paramount for these predictions, more specifically regarding gapless energetic maps. This work aims to eliminate the energetic map’s gaps using two Machine Learning (ML) approaches: regression and classification. The proposed ML solution builds upon the synergy between map-information and crowdsourced driving profiles of 4.6 million kilometres of training and test traces. For evaluation, two test-scenarios capture the models’ performance for the analysed problem in two perspectives. First, we evaluate our ML models, followed by the problem-specific energetic evaluation perspective for better interpretability. From the latter, the results indicate energetic map data imputation performs promisingly better when using the regression instead of the classification model.

show abstract

“…Sensor failure, transmission network failures, and environmental factors often lead to generate various data quality problems (incompleteness, error, noise, etc.) [15,16]. In order to detect and recover the abnormal data while improving the data quality, in this paper, a novel bidirectional searching strategy was developed based on the k-nearest neighbor (KNN) approach.…”

Section: Introductionmentioning

confidence: 99%

A Bidirectional Searching Strategy to Improve Data Quality Based on K-Nearest Neighbor Approach

Liang

Qin

2019

Symmetry

View full text Add to dashboard Cite

Traffic data are the basis of traffic control, planning, management, and other implementations. Incomplete traffic data that are not conducive to all aspects of transport research and related activities can have adverse effects such as traffic status identification error and poor control performance. For intelligent transportation systems, the data recovery strategy has become increasingly important since the application of the traffic system relies on the traffic data quality. In this study, a bidirectional k-nearest neighbor searching strategy was constructed for effectively detecting and recovering abnormal data considering the symmetric time network and the correlation of the traffic data in time dimension. Moreover, the state vector of the proposed bidirectional searching strategy was designed based the bidirectional retrieval for enhancing the accuracy. In addition, the proposed bidirectional searching strategy shows significantly more accuracy compared to those of the previous methods.Symmetry 2019, 11, 815 2 of 18 and model parameter selection can be shown. In Section 4, basic KNN approach will be introduced. In Section 5, a novel bidirectional data recovery approach is proposed, and the parameter setting process is presented. Section 6 discusses the experimental design and the results. Section 7 concludes the paper with a summary of bidirectional searching strategy and gives the suggestions for future work. Literature ReviewSeveral works focusing on recovering the abnormal data response to the data quality control strategy. These studies examine the effect of abnormal data and research the data recovery approach based on the historical aiming to improve data quality. Pushkar et al. [17] applied the catastrophe theory to establish the three-parameter sudden change surface of traffic flow to recover data and then proposed the speed estimation method. A nearest-neighbor imputation algorithm was developed and applied to interpolate the missing data based on the average value of historical data at the same time interval [18]. In addition, the factor approach estimated the missing data by using the mean value of the key factors selected from the historical set [19]. Troyanskaya et al. [20] investigated automated methods for estimating missing data to minimize the effect of incomplete data sets on analyses. Smith et al. [21] performed a preliminary analysis of several heuristic and statistical imputation techniques and declared the statistical techniques are more accurate. Chen et al. [22] proposed a linear regression algorithm to impute missing or bad traffic flow data and occupancy data using neighboring sensors data. Abdella et al. [23] proposed an integrated method combining the genetic algorithms with neural networks aiming at seeking the approximating missing data in a database. Tang et al.[24] developed a hybrid approach integrating Fuzzy C-Means-based (FCM) imputation method with a genetic algorithm (GA) to estimate the missing traffic volume data based on inductance loop sensor outputs. This approach ...

show abstract

On the imputation of missing data for road traffic forecasting: New insights and novel techniques

Cited by 88 publications

References 35 publications

Analysis and Impact Evaluation of Missing Data Imputation in Day-ahead PV Generation Forecasting

Analysis and Impact Evaluation of Missing Data Imputation in Day-ahead PV Generation Forecasting

Energetic Map Data Imputation: A Machine Learning Approach

A Bidirectional Searching Strategy to Improve Data Quality Based on K-Nearest Neighbor Approach

Contact Info

Product

Resources

About