Approaches for assessing performance of high-resolution mass spectrometry–based non-targeted analysis methods

Abstract

Non-targeted analysis (NTA) using high-resolution mass spectrometry has enabled the detection and identification of unknown and unexpected compounds of interest in a wide range of sample matrices. Despite these benefits of NTA methods, standardized procedures do not yet exist for assessing performance, limiting stakeholders’ abilities to suitably interpret and utilize NTA results. Herein, we first summarize existing performance assessment metrics for targeted analyses to provide context and clarify terminology that may be shared between targeted and NTA methods (e.g., terms such as accuracy, precision, sensitivity, and selectivity). We then discuss promising approaches for assessing NTA method performance, listing strengths and key caveats for each approach, and highlighting areas in need of further development. To structure the discussion, we define three types of NTA study objectives: sample classification, chemical identification, and chemical quantitation. Qualitative study performance (i.e., focusing on sample classification and/or chemical identification) can be assessed using the traditional confusion matrix, with some challenges and limitations. Quantitative study performance can be assessed using estimation procedures developed for targeted methods with consideration for additional sources of uncontrolled experimental error. This article is intended to stimulate discussion and further efforts to develop and improve procedures for assessing NTA method performance. Ultimately, improved performance assessments will enable accurate communication and effective utilization of NTA results by stakeholders. Graphical abstract

Influence of data acquisition modes and data analysis approaches on non-targeted analysis of phthalate metabolites in human urine

“…The NTA community has currently devoted many efforts to standardize these workflows and provide guidelines for reporting NTA methods and results, which aims to be able to provide comparable qualitative and quantitative information for exposure and risk assessment [25][26][27][28]. However, these new methodological approaches still face a number of limitations and challenges [29].…”

Development and Implementation of Automated Qualification Processes for the Identification of Pollutants in an Aquatic Environment from High-Resolution Mass Spectrometric Nontarget Screening Data

“…Details on the data treatment are provided in SI 3. General information about the treatment of nontarget screening data and classification aspects have been provided by Fisher et al and can help facilitate the comprehension of the results of this study.…”

“…Details on the data treatment are provided in SI 3. General information about the treatment of nontarget screening data and classification aspects have been provided by Fisher et al 25 This constitutes a binary classification problem, which can be solved using different approaches. One particular method is to use machine learning based on common properties, so-called features, to provide an answer to this challenge with a confidence value associated.…”

Evaluating non-targeted analysis methods for chemical characterization of organic contaminants in different matrices to estimate children’s exposure

“…Selectivity was assessed by the true negative rate (TNR), which is calculated based on the following equation: TNR = TN/(FP + TN), in which FP are false positive compounds (compounds falsely identified as being present when they are not) and TN are the compounds that are not present and are correctly rejected [ 25 ]. Accuracy in our NTA approach was calculated using the formula (TP + TN)/(TP + FP + FN + TN) which calculates the methods ability to correctly identify both TP and TN [ 25 ]. Precision of the NTA method was evaluated using the formula TP/(TP + FP) which is the method’s ability to identify compounds correctly in relation to false identifications [ 25 ].…”

“…Accuracy in our NTA approach was calculated using the formula (TP + TN)/(TP + FP + FN + TN) which calculates the methods ability to correctly identify both TP and TN [ 25 ]. Precision of the NTA method was evaluated using the formula TP/(TP + FP) which is the method’s ability to identify compounds correctly in relation to false identifications [ 25 ].…”