Methods for pre-processing smartcard data to improve data quality

Robinson, Steve; Narayanan, Baskaran; Toh, Nelson; Pereira, Francisco C.

doi:10.1016/j.trc.2014.10.006

Cited by 49 publications

(23 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…There are many errors (e.g., system error, user error, device error) which may generate in the data collection process [31], therefore it's vital to assess smart collection data quality. Three indicators are taken into consideration, namely, sample size, whether data cleaning and data description is done.…”

Section: Quality Assessmentmentioning

confidence: 99%

See 1 more Smart Citation

Smart Card Data Mining of Public Transport Destination: A Literature Review

Sun

Jing

et al. 2018

Information

View full text Add to dashboard Cite

Smart card data is increasingly used to investigate passenger behavior and the demand characteristics of public transport. The destination estimation of public transport is one of the major concerns for the implementation of smart card data. In recent years, numerous studies concerning destination estimation have been carried out-most automatic fare collection (AFC) systems only record boarding information but not passenger alighting information. This study provides a comprehensive review of the practice of using smart card data for destination estimation. The results show that the land use factor is not discussed in more than three quarters of papers and sensitivity analysis is not applied in two thirds of papers. In addition, the results are not validated in half the relevant studies. In the future, more research should be done to improve the current model, such as considering additional factors or making sensitivity analysis of parameters as well as validating the results with multi-source data and new methods.

show abstract

Section: Quality Assessmentmentioning

confidence: 99%

“…Evidence showed that the AFC system transaction data often has some potential problems during data collecting, which may be caused by software, erroneous data, faulty hardware or the users [31]. The main existing data defections and processing methods are shown in Table 7.…”

Section: Data Preparationmentioning

confidence: 99%

Smart Card Data Mining of Public Transport Destination: A Literature Review

Sun

Jing

et al. 2018

Information

View full text Add to dashboard Cite

show abstract

“…The dataset for three weekdays and one weekend day from the South East Queensland SEQ bus, train, and ferry modes are selected. Wednesday to Saturday (20)(21)(22)(23) March 2013) are chosen as the weather on all four days was normal, and there were no special events during those days. 20,000 passengers randomly are selected for each day, who approximately make 45,000 trip legs per day.…”

Section: Resultsmentioning

confidence: 99%

“…The dataset first needs to be cleaned [20]. A trip leg is a distance and period between a consecutive boarding and alighting transactions of a passenger.…”

Section: Methodsmentioning

confidence: 99%

Spatial-Temporal Similarity Correlation between Public Transit Passengers Using Smart Card Data

Faroqi

Mesbah

Kim

2017

Journal of Advanced Transportation

View full text Add to dashboard Cite

The increasing availability of public transit smart card data has enabled several studies to focus on identifying passengers with similar spatial and/or temporal trip characteristics. However, this paper goes one step further by investigating the relationship between passengers' spatial and temporal characteristics. For the first time, this paper investigates the correlation of the spatial similarity with the temporal similarity between public transit passengers by developing spatial similarity and temporal similarity measures for the public transit network with a novel passenger-based perspective. The perspective considers the passengers as agents who can make multiple trips in the network. The spatial similarity measure takes into account direction as well as the distance between the trips of the passengers. The temporal similarity measure considers both the boarding and alighting time in a continuous linear space. The spatial-temporal similarity correlation between passengers is analysed using histograms, Pearson correlation coefficients, and hexagonal binning. Also, relations between the spatial and temporal similarity values with the trip time and length are examined. The proposed methodology is implemented for four-day smart card data including 80,000 passengers in Brisbane, Australia. The results show a nonlinear spatial-temporal similarity correlation among the passengers.

show abstract

“…Using smart card data as a new source of data collection can take advantage of the large and continuous dataset. By using smart card data, the user's role in previous data collection via the survey process is minimized, which improves data quality (Bagchi and White, 2004;Bagchi and White, 2005;Robinson et al, 2014).…”

Section: Increase Data Quality and Quantitymentioning

confidence: 99%

Estimation of transit origin destination matrices using smart card fare data

Alsger¹

View full text Add to dashboard Cite

Smart card fare systems have become a valuable source of information for public transport OriginDestination (O-D) estimation, allowing a better understanding of individual travel patterns and improving strategic public transport planning. The O-D matrix is important for transportation analysis, design, and management. It gives indispensable information on the travel demand between two different locations, which is used in many transportation applications from strategic planning to traffic control and management.High quality traffic information is required to improve the estimation of O-D matrix in public transportation. Many transit agencies around the world are now using smart card systems to replace traditional payment methods as a viable payment option. Furthermore, the smart card improves the quality of the data; increases the amount of statistics available; updates data continuously 24 hours per day; reduces boarding and alighting time and the driver's workload; saves costs in data collection and editing, eliminates human errors; and provides new opportunities for innovative and flexible fare structuring.Most Automated Fare Collection (AFC) systems, especially when implemented on buses, record passenger boarding information; however, no alighting information is recorded. The lack of alighting stop details is the result of the installed AFC systems, where passengers are not required to use their cards when alighting. Given the information limitations of many AFC systems, the accuracy level of the estimated O-D matrices is often unknown. In addition, many of the assumptions made in suchmethods are yet to be tested.Although smart card fare data has attracted consideration attention as a rich and comprehensive source of information, it has some limitations which constrain its application. One of these limitations is the missing information of passengers' trip purpose. The main reason for the missing information is due to its original use being the collection of revenue and not data collection. The smart card data analysed in the current research was obtained from Translink, the public transport authority of South East Queensland (SEQ), Australia. An important aspect of this system is that it includes both boarding and alighting times and locations, where a passenger gets on or off a public transport vehicle.The first stage of the current research deals with implementing, validating and improving the tripchaining method and its assumptions to estimate public transport O-D matrices using smart card fare data. Transfer time threshold, transfer walking distance, and the location of the last destination of a passenger in a given day, are the major assumptions investigated here. The available rich and unique ii smart card fare data allowed the current investigation and validation of the effect of different tripchaining method assumptions on the estimated matrices. The estimated O-D results are validated with the actual matrices, which are based on full boarding and alighting information. The errors distributi...

show abstract

Methods for pre-processing smartcard data to improve data quality

Cited by 49 publications

References 15 publications

Smart Card Data Mining of Public Transport Destination: A Literature Review

Smart Card Data Mining of Public Transport Destination: A Literature Review

Spatial-Temporal Similarity Correlation between Public Transit Passengers Using Smart Card Data

Estimation of transit origin destination matrices using smart card fare data

Contact Info

Product

Resources

About