The sbv IMPROVER Systems Toxicology computational challenge: Identification of human and species-independent blood response markers as predictors of smoking exposure and cessation status

Abstract

Cigarette smoking entails chronic exposure to a mixture of harmful chemicals that trigger molecular changes over time, and is known to increase the risk of developing diseases. Risk assessment in the context of 21st century toxicology relies on the elucidation of mechanisms of toxicity and the identification of exposure response markers, usually from high-throughput data, using advanced computational methodologies. The sbv IMPROVER Systems Toxicology computational challenge (Fall 2015–Spring 2016) aimed to evaluate whether robust and sparse (≤40 genes) human (sub-challenge 1, SC1) and species-independent (sub-challenge 2, SC2) exposure response markers (so called gene signatures) could be extracted from human and mouse blood transcriptomics data of current (S), former (FS) and never (NS) smoke-exposed subjects as predictors of smoking and cessation status. Best-performing computational methods were identified by scoring anonymized participants’ predictions. Worldwide participation resulted in 12 (SC1) and six (SC2) final submissions qualified for scoring. The results showed that blood gene expression data were informative to predict smoking exposure (i.e. discriminating smoker versus never or former smokers) status in human and across species with a high level of accuracy. By contrast, the prediction of cessation status (i.e. distinguishing FS from NS) remained challenging, as reflected by lower classification performances. Participants successfully developed inductive predictive models and extracted human and species-independent gene signatures, including genes with high consensus across teams. Post-challenge analyses highlighted “feature selection” as a key step in the process of building a classifier and confirmed the importance of testing a gene signature in independent cohorts to ensure the generalized applicability of a predictive model at a population-based level. In conclusion, the Systems Toxicology challenge demonstrated the feasibility of extracting a consistent blood-based smoke exposure response gene signature and further stressed the importance of independent and unbiased data and method evaluations to provide confidence in systems toxicology-based scientific conclusions.

A Meta-Analysis of the Performance of a Blood-Based Exposure Response Gene Signature Across Clinical Studies on the Tobacco Heating System 2.2 (THS 2.2)

“…This smoke exposure response signature (SERS) of 11 genes ( LRRN3, SASH1, PALLD, RGL1, TNFRSF17, CDKN1C, IGJ, RRM2, ID3, SERPING1 , and FUCA1 ) has since been verified in the systems biology verification Industrial Methodology for PROcess VErification in Research (sbv IMPROVER) Systems Toxicology Computational Challenge. The genes proposed by the best performers overlapped remarkably with our gene signature (Belcastro et al, 2017; Poussin et al, 2017). The SERS could distinguish CS from NS with remarkable accuracy (Martin et al, 2015).…”

“…Furthermore, the prediction scores from the training and validation cohort showed a clear separation between the FS (NS, respectively) smokers and the CS, allowing for a quantitative monitoring of smoking exposure. In these studies, no separation between FS (at least one year after quitting smoking) and NS was observed (Belcastro et al, 2017; Poussin et al, 2017).…”

The product science of electrically heated tobacco products: a narrative review of the scientific literature

“…These conclusions are in agreement with those of a separate review published by PMI researchers, which concluded that "the experimental results [from systems biology and systems toxicology studies] demonstrate a reduced impact on apical and molecular endpoints, no novel effect not seen with cigarette smoke exposure, and an effect of switching from cigarettes to either MRTP [two heated tobacco products; one electrically heated and one with a carbon-heating design] that is comparable to that of complete smoking cessation" (Schlage et al, 2020). PMI researchers have also attempted to further validate their scientific approach to assessment of the THS2.2 heated tobacco product by providing their biological and clinical samples or data packages to external independent researchers (in a blinded fashion) to see whether or not they would reach the same conclusions as the original researchers (Poussin et al, 2017;Boué et al, 2019;Belcastro et al, 2020). In each instance reported, the external researchers corroborated with the conclusions reached by the original researchers based on their own independent analyses of the samples.…”

Crowd-Sourced Verification of Computational Methods and Data in Systems Toxicology: A Case Study with a Heat-Not-Burn Candidate Modified Risk Tobacco Product

“…The quality check excluded CEL files that did not pass minimum quality criteria. 34,35 To facilitate data set handling, human and mouse gene expression data sets were provided with human gene symbols for both. Mouse genes were orthologized to human genes using the HGNC/HCOP mapping database (downloaded on 11 Dec, 2014).…”

“…Raw data (CEL files) from each data set were processed and normalized in the R environment (v3.1.2) using frozen Robust Microarray Analysis, fRMA v1.1. , The custom Brainarray CDF files for mice and humans were used in the Affymetrix probe to Entrez Gene ID mapping, resulting in one probe set for one gene relationship (HGU133Plus2_Hs_ENTREZG v16.0, Mouse4302_Mm_ENTREZG v16.0, respectively). The quality check excluded CEL files that did not pass minimum quality criteria. , To facilitate data set handling, human and mouse gene expression data sets were provided with human gene symbols for both. Mouse genes were orthologized to human genes using the HGNC/HCOP mapping database (downloaded on 11 Dec, 2014) .…”