Exploring Imputation Techniques for Missing Data in Transportation Management Systems

Abstract

Many states have implemented large-scale transportation management systems to improve mobility in urban areas. These systems are highly prone to missing and erroneous data, which results in drastically reduced data sets for analysis and real-time operations. Imputation is the practice of filling in missing data with estimated values. Currently, the transportation industry generally does not use imputation as a means for handling missing data. Other disciplines have recognized the importance of addressing missing data and, as a result, methods and software for imputing missing data are becoming widely available. The feasibility and applicability of imputing missing traffic data are addressed, and a preliminary analysis of several heuristic and statistical imputation techniques is performed. Preliminary results produced excellent performance in the case study and indicate that the statistical techniques are more accurate while maintaining the natural characteristics of the data.

Creating Data Outputs from Multi Agent Traffic Micro Simulation to Assimilation with Real Traffic Data

“…Imputation techniques developed thus far can be classified into three categories: regression, nearest neighbor and deck replacement, and classifi cation. Their applications in traffic data imputation have been presented by Smith et al in [7], Al-Deek and Chandra in [8] and Gold et al in [9]. Researchers in [10] consider eleven algorithms for imputing missing traffic data recorded by automatic loop detectors in the Dallas, Texas region.…”

Weigh-In-Motion Data Checking and Imputation

“…Average errors for refined models were lower than 1 percent, and the 95th percentile errors were below 2 percent for counts with stable patterns. Smith et al (2003) provided a comparison of heuristic techniques and statistical techniques and indicated that the more sophisticated statistical techniques may generate better imputations. Smith et al also suggested that the transportation profession seriously reconsider the AASHTO "policy" of not imputing traffic data in order to provide the user as much information as possible.…”

Developing a Strategy for Imputing Missing Traffic Volume Data

“…Smith et al (2003) analyzed such methods as average of surrounding detectors, average of surrounding time periods, historic average, and factor up. More sophisticated statistical procedures, such as expectation maximization and data augmentation, were also examined.…”

“…Their applications in traffi c data imputation have been presented by Smith et al (2003), Al-Deek and Chandra (2004), and Gold et al (2001) for archived data management systems in Virginia, Florida, and Texas. In addition, Zhong et al (2004) discussed the application of neural networks and a genetic algorithm for imputing missing traffi c data from permanent count stations.…”