A machine learning calibration model using random forests to improve sensor performance for lower-cost air quality monitoring

Abstract

Abstract. Low-cost sensing strategies hold the promise of denser air quality monitoring networks, which could significantly improve our understanding of personal air pollution exposure. Additionally, low-cost air quality sensors could be deployed to areas where limited monitoring exists. However, low-cost sensors are frequently sensitive to environmental conditions and pollutant cross-sensitivities, which have historically been poorly addressed by laboratory calibrations, limiting their utility for monitoring. In this study, we investigated different calibration models for the Real-time Affordable Multi-Pollutant (RAMP) sensor package, which measures CO, NO 2 , O 3 , and CO 2 . We explored three methods: (1) laboratory univariate linear regression, (2) empirical multiple linear regression, and (3) machine-learning-based calibration models using random forests (RF). Calibration models were developed for 16-19 RAMP monitors (varied by pollutant) using training and testing windows spanning August 2016 through February 2017 in Pittsburgh, PA, US. The random forest models matched (CO) or significantly outperformed (NO 2 , CO 2 , O 3 ) the other calibration models, and their accuracy and precision were robust over time for testing windows of up to 16 weeks. Following calibration, average mean absolute error on the testing data set from the random forest models was 38 ppb for CO (14 % relative error), 10 ppm for CO 2 (2 % relative error), 3.5 ppb for NO 2 (29 % relative error), and 3.4 ppb for O 3 (15 % relative error), and Pearson r versus the reference monitors exceeded 0.8 for most units. Model performance is explored in detail, including a quantification of model variable importance, accuracy across different concentration ranges, and performance in a range of monitoring contexts including the National Ambient Air Quality Standards (NAAQS) and the US EPA Air Sensors Guidebook recommendations of minimum data quality for personal exposure measurement. A key strength of the RF approach is that it accounts for pollutant cross-sensitivities. This highlights the importance of developing multipollutant sensor packages (as opposed to single-pollutant monitors); we determined this is especially critical for NO 2 and CO 2 . The evaluation reveals that only the RF-calibrated sensors meet the US EPA Air Sensors Guidebook recommendations of minimum data quality for personal exposure measurement. We also demonstrate that the RF-model-calibrated sensors could detect differences in NO 2 concentrations between a near-road site and a suburban site less than 1.5 km away. From this study, we conclude that combining RF models with carefully controlled state-of-the-art multipollutant sensor packages as in the RAMP monitors appears to be a very promising approach to address the poor performance that has plagued low-cost air quality sensors.

Field evaluation of low-cost particulate matter sensors in high and low concentration environments

“…Metrics such as the intercept, slope, and coefficient of determination (R 2 ) obtained from OLS models of sensor outputs with reference instrument measurements are widely used to evaluate sensor performance (Holstius et al, 2014;Gao et al, 2015;Wang et al, 2015;Jiao et al, 2016;Cross et al, 2017;Kelly et al, 2017;Zimmerman et al, 2018). In this study, all the R 2 in…”

“…To date, only a few studies have attempted to 10 measure parameters other than R 2 to gauge the overall performance of low-cost sensor technologies. They typically focus on the RMSE (Holstius et al, 2014;Cross et al, 2017;Zimmerman et al, 2018), the mean absolute error (MAE) and the mean bias error (MBE) (Cross et al, 2017;Zimmerman et al, 2018), and normalized residuals (Sousan et al, 2017;Kelly et al, 2017). In addition to the intercept, slope, and R 2 , we also used ratios of the calibrated PMS3003 PM2.5 mass concentrations to reference monitor values to examine sensors' post-calibration performance.…”

Assessing a low-cost methane sensor quantification system for use in complex rural and urban environments

“…Given the variety of low-cost sensors available, using the Figaro TGS 2600 sensors in a sensor array could provide additional signals at each deployment site facilitating more reliable data. Including multiple sensor signals in a neural network calibration approach may also improve the accuracy of the calibrated data (Zimmerman et al, 2018;De Vito et al, 2008;Huyberechts et al, 1997). Future analysis of the data collected in Los Angeles and continued use of this sensor in areas with complex mixtures will require carbon monoxide and non-methane hydrocarbon impacts be considered.…”

“…Several studies have also made use of sensors to study the spatial variability of O 3 on various scales (Sadighi et al, 2018;Cheadle et al, 2017;Moltchanov et al, 2015). Connected to this effort on sensor applications, there has been much work evaluating the performance of individual sensors (Masson et al, 2015a, b;Spinelle et al, 2015Spinelle et al, , 2017Lewis at al., 2016) and demonstrating the performance of different calibration approaches (Zimmerman et al, 2018;Kim et al, 2018;Cross et al, 2017).…”

Atmospheric particulate matter characterization by Fourier Transform Infrared spectroscopy: a review of statistical calibration strategies for carbonaceous aerosol quantification in US measurement networks

“…While partial least squares regression and its variants figure heavily in the calibration approach taken thus far, related developments in the fields of machine learning, chemometrics, and statistical process monitoring are mentioned to indicate the range of possibilities yet available to overcome future challenges in interpreting complex the mid-IR spectra of PM. We expect that many concepts described here will also be relevant for the emerging field of statistical 10 calibration and deployment of measurements in a broader environmental and atmospheric context (e.g., Cross et al, 2017;Kim et al, 2017;Zimmerman et al, 2018). In the following sections, we describe the experimental methods for collecting data (Section 2), the calibration process (Section 3), assessing suitability of existing models for new samples (Section 4.1), and maintaining calibration models (Section 4.2).…”