Imputing Missing Data in Hourly Traffic Counts

Abstract

Hourly traffic volumes, collected by automatic traffic recorders (ATRs), are of paramount importance since they are used to calculate average annual daily traffic (AADT) and design hourly volume (DHV). Hence, it is necessary to ensure the quality of the collected data. Unfortunately, ATRs malfunction occasionally, resulting in missing data, as well as unreliable counts. This naturally has an impact on the accuracy of the key parameters derived from the hourly counts. This study aims to solve this problem. ATR data from New South Wales, Australia was screened for irregularities and invalid entries. A total of 25% of the reliable data was randomly selected to test thirteen different imputation methods. Two scenarios for data omission, i.e., 25% and 100%, were analyzed. Results indicated that missForest outperformed other imputation methods; hence, it was used to impute the actual missing data to complete the dataset. AADT values were calculated from both original counts before imputation and completed counts after imputation. AADT values from imputed data were slightly higher. The average daily volumes when plotted validated the quality of imputed data, as the annual trends demonstrated a relatively better fit.

Missing Traffic Data Imputation for Artificial Intelligence in Intelligent Transportation Systems: Review of Methods, Limitations, and Challenges

“…With regard to learning methods, predictive mean matching (PMM) based on multiple imputations by chained equations (MICE) has been looked into in [63]. A study done later on has then proceeded to compare variations of PMM methods, including MICE, Classification and Regression Trees (CART), Least Absolute Shrinkage and Selection Operator (LASSO), and random forest, with the result being the MissForest implementation of Random Forest being the best performer [11]. It is noteworthy that random forest is considered a machine learning algorithm, which shows why machine learning tends to be researched more compared to statistical methods, especially in recent years.…”

“…Other missing data review papers, such as [10], have made a comparison between different missing traffic data imputation methods, namely prediction, interpolation, and statistical learning methods, and concluded that the PPCA-based (Probability Principle Component Analysis) methods perform the best overall in terms of accuracy and computational complexity. In addition, [11] has compared the performance of variations of other existing statistical methods such as linear regressions, Predicting Mean Matching (PMM), and mean imputation, while also comparing regression tree-based methods such as Classification and Regression Trees (CART) and Random Forest. The conclusion is that the random forest implementation performed the best.…”

“…Also, the authors would like to note that many public datasets, such as PeMS [12], are freeway traffic datasets, which do not equate to an urban traffic environment, as also mentioned by [5] and [4]. Certain studies may make use of road segments in an urban environment [11], but a few individual roads are not representative of urban traffic as a whole. In reality, urban networks are more likely in need of such research, and this paper aims to review recent papers whose work covers urban networks.…”