Performance of continuous emission monitoring solutions under single-blind controlled testing protocol.

Abstract

Continuous emission monitoring (CM) solutions promise to accelerate fugitive methane emission mitigation in natural gas infrastructure by detecting large sources sooner than they would be detected in traditional leak surveys. Quantification by CM solutions has also been proposed as the foundation of measurement-based inventories. Herein we present the first published results of CM solutions tested using a consensus protocol. Single-blind testing was performed with controlled releases at a dedicated facility, using test conditions that were challenging but less complex than typical field conditions. Eleven solutions were tested, including point sensor networks and scanning/imaging solutions. Results indicate detection limits (emission rate with 90% probability of detection) of 3-30 kg CH4/h, with 6 of 11 solutions in the range of 3-6 kg CH4/h, with high uncertainty. Of the 11 solutions tested, 6 provided emission rates estimates and for emission rate of 0.1-1 kg/h, the solutions' mean relative errors ranged from -44% to +586% with single estimates between -90% and +300% (all uncertainties stated as empirical 95% confidence intervals) with most solutions' upper uncertainty exceeding +900%. Solutions' mean relative error for emission rates >1 kg/h is -40% to +93% with two solutions achieving mean relative errors below 20%. In this range, all solutions exhibited single-estimate relative error from below -80% to values above +240%. When test results were simulated using emissions data from a portfolio of facilities, two solutions estimated emissions to within ±20%, while 4 estimated emissions at least double the true emission rate. The large variability in performance between CM solutions, coupled with highly uncertain detection, detection limit, and quantification results, indicate that the performance of individual CM solutions should be well understood before relying on results for internal emissions mitigation programs or regulatory reporting. Additionally, given the rapid development of these solutions, additional implementation standards and regular testing will also be necessary.

Single-Blind Detection, Localization, and Quantification of Methane Emissions Using Continuous Path-Integrated Column Measurements

“…While the CMP metrics are a useful framework for providing a method to compare the key characteristics of continuous monitoring systems that are most of interest to oil and gas operators and regulators, our work illuminates some refinements that could be made to further increase the utility of the metrics. Limitations of the CMP methodology to perform classification of detections as TP or FP detections have been identified in other testing programs, 11 suggesting a need to consider the application or use case of the continuous monitoring system under evaluation and to tailor the methodology as appropriate. Defining FPF such that it is not dependent on the number of controlled releases would allow more meaningful comparison between systems tested under very different experimental designs.…”

“…Recent evaluation of continuous monitoring systems at METEC included hundreds of individual releases, some lasting up to 8 hours, at emission rates up to 6.39 kg/h. 11 In this study, releases were limited to rates ≤ 2.15 kg/h and durations ≤ 60 minutes. Initial unblind test results indicated that high release rates were not needed to identify the minimum detection threshold and that short durations would provide ample information on detection thresholds.…”

“…More recent testing at METEC utilizing the CMP has addressed many of these methodological issues, with participating solutions fully blind and greater numbers of controlled releases. 11 The GreenLITE™ experiment detailed here allowed for releases anywhere within 20 grid boxes covering a total area of over 0.18 km 2 at any time within a 12-hour daily window on any of 27 days. The longest release lasted one hour, and the average release was 22.8 minutes in duration.…”

Point Sensor Networks Struggle to Detect and Quantify Short Controlled Releases at Oil and Gas Sites

“…During the eld campaign, the solutions did not provide detection reports using the same email based reporting method as required during METEC testing. 19 Instead, the study team was granted access to the solution's "dashboard", a graphical user interface provided to operators to receive alerts, interact with data, investigate or acknowledge detections, and The original intent of the eld campaign was to utilize the same metrics as METEC CR testing, specically POD, quantication accuracy, and localization accuracy. 19 Since solutions did not provide dened detection reports that could be used for this purpose, the eld team needed to interpret the dashboards' raw data to determine if there was a sucient change in emissions that the presence of excess emissions at the site could be reasonably identied.…”

Evaluating Development of Empirical Estimates Using Two Top-Down Methods at Midstream Natural Gas Facilities

“…A major investment by the Advanced Research Project Agency-Energy (ARPA-E) has stimulated substantial commercial investment over the last decade in measurement methods to assess emissions using remote sensing techniques, including drones, manned aircraft, and satellite technologies. The increase in investment and method development have been accompanied by extensive testing programs using a wide variety of protocols [10,11], as well as a major Department of Energy-funded program [12,13] to develop standardized test protocols.…”