Comparative Evaluation of in Silico Systems for Ames Test Mutagenicity Prediction: Scope and Limitations

Abstract

The predictive power of four commonly used in silico tools for mutagenicity prediction (DEREK, Toxtree, MC4PC, and Leadscope MA) was evaluated in a comparative manner using a large, high-quality data set, comprising both public and proprietary data (F. Hoffmann-La Roche) from 9,681 compounds tested in the Ames assay. Satisfactory performance statistics were observed on public data (accuracy, 66.4-75.4%; sensitivity, 65.2-85.2%; specificity, 53.1-82.9%), whereas a significant deterioration of sensitivity was observed in the Roche data (accuracy, 73.1-85.5%; sensitivity, 17.4-43.4%; specificity, 77.5-93.9%). As a general tendency, expert systems showed higher sensitivity and lower specificity when compared to QSAR-based tools, which displayed the opposite behavior. Possible reasons for the performance differences between the public and Roche data, relating to the experimentally inactive to active compound ratio and the different coverage of chemical space, are thoroughly discussed. Examples of peculiar chemical classes enriched in false negative or false positive predictions are given, and the results of the combined use of the prediction systems are described.

On the Use of In Silico Tools for Prioritising Toxicity Testing of the Low‐Volume Industrial Chemicals in REACH

“…1C); however, combing of structural alerts increases the number of false positives of already conservative rule-based models (Toxtree and Derek Nexus). This is consistent with previous findings that rulebased models yield large numbers of false positives [16,27]. Conservative predictions were also found for a hybrid model (VEGA) which uses both statistical methods and structural alerts.…”

“…3). The common false negatives were found to be small compounds with relatively low complexity which agrees with systematic misclassifications described previously [16]. Most of the rulebased expert systems did not identify any structural alerts associated with these compounds, and it is known that there is a lack of structural alerts for non-genotoxic carcinogens [25] (here four of nine C-chemicals are non-genotoxic carcinogens).…”

“…A particular focus is put on models that could provide information indicating whether the substance could be classified as carcinogenic, mutagenic or toxic to reproduction (CMR) according to Dangerous Substances Directive [9] or CLP Regulation [10]. To our knowledge, previous studies on the utility of non-experimental approaches to predicting CMR properties focused on comparing the results of specific assays to those obtained using selected in silico models rather than the use of in silico results for CLP classification and the prioritisation of industrial compounds for further testing according to the REACH legislation [11][12][13][14][15][16][17][18][19][20][21].…”

The Use of In Silico Models Within a Large Pharmaceutical Company

“…A number of factors contributed to the increased importance of in silico methods in drug discovery: (1) wider avail-ability of high-quality datasets (public domain, focused data sharing initiatives), (2) robust computational models that can provide reliable predictions [ 9 ], (3) pressure to reduce animal testing, (4) need to bring new drugs to the market faster and cheaper, (5) legislation on the assessment of potential genotoxic impurities, and (6) greater number of commercially available and open-source software tools.…”

Prediction of genotoxic potential of cosmetic ingredients by an in silico battery system consisting of a combination of an expert rule-based system and a statistics-based system

“…Already, in silico models have been used for prediction of organ-specific toxicities, such as hepatotoxicity (Contrera et al, 2003), cardiotoxicity (Pearlstein et al, 2003), and nephrotoxicity (Myshkin et al, 2012), as well as for prediction of biochemical toxicities, such as plasma protein binding (Li et al, 2011), cytochrome P450 inhibition (Wanchana et al, 2003), blood-brain barrier permeability (Martins et al, 2012), and ADME (Hosea and Jones, 2013). They are also being used for prediction of genomic toxicities, such as genotoxicity and carcinogenicity (Hillebrecht et al, 2011), and reproductive and developmental toxicity (Worth et al, 2011). The International Conference on Harmonization (ICH) M7 Guideline on Assessment and Control of DNA Reactive (Mutagenic) Impurities in Pharmaceuticals to Limit Potential Carcinogenic Risk reached step 4 of the ICH Process in June 2014 (ICH, 2014).…”