Revisiting inconsistency in large pharmacogenomic studies

Abstract

Background : In 2012, two large pharmacogenomic studies, the Genomics of Drug Sensitivity in Cancer (GDSC) and Cancer Cell Line Encyclopedia (CCLE), were published, each reported gene expression data and measures of drug response for a large number of drugs and hundreds of cell lines. In 2013, we published a comparative analysis that reported gene expression profiles for the 471 cell lines profiled in both studies and dose response measurements for the 15 drugs characterized in the common cell lines by both studies. While we found good concordance in gene expression profiles, there was substantial inconsistency in the drug responses reported by the GDSC and CCLE projects. Our paper was widely discussed and we received extensive feedback on the comparisons that we performed. This feedback, along with the release of new data, prompted us to revisit our initial analysis. Here we present a new analysis using these expanded data in which we address the most significant suggestions Consistency of drug sensitivity dataWe began by computing the area between the two drug doseresponse curves (ABC) to assess consistency of cell viability data for each cell line combination screened in both GDSC and CCLE using the common concentration range. ABC measures the difference between two drugdose response curves by estimating the absolute area between these curves, which ranges from 0% (perfect consistency) to 100% (perfect inconsistency). The ABC statistic identified highly consistent ( Figure 4A,B) and highly inconsistent ( Figure 4C,D) doseresponse curves between GDSC and CCLE. The mean of the ABC estimates for all drugcell line combinations was 10% ( Supplementary Figure 2A), with PD0332991 yielding the highest discrepancies (Supplementary Figure 2B).We compared biological replicates in GDSC, which were performed independently at the Massachusetts General Hospital (MGH) and the Wellcome Trust Sanger Institute (WTSI). These experiments are comprised of 577 cell lines treated with AZD6482, a PI3Kβ inhibitor screened in GDSC ( Supplementary File 4). We computed the ABC of these biological replicates and observed both highly consistent and inconsistent cases ( Supplementary Figure 3). We then computed the median ABC values for each pair of drugs in GDSC and used these as a distance metric for complete linkage hierarchical clustering. We found that the MGH and WTSIadministered AZD6482 experiments clustered together, suggesting that the differences between doseresponse curves of biological replicates were smaller than the differences observed between different drugs (Supplementary Figure 4A). We performed the same clustering analysis by computing the ABCbased distance between all the drugs in GDSC and CCLE and observed that only three out of the fifteen common drugs clustered tightly (17AAG, lapatinib, and PHA−665752; Supplementary Figure 4B). Despite the small number of cell lines exhibiting sensitivity to PHA−665752 and lapatinib, these drugs closely clustered between GDSC and CCLE; however this was not the case for other highly ta...

Assessment of pharmacogenomic agreement

“…Our stringent reanalysis of the most updated data from the GDSC and CCLE confirms our 2013 finding that the measures of drug response reported by these two groups are not consistent and have not improved substantially as the groups have continued generating data since 2012 5 . While the authors make arguments suggesting consistency, it is difficult to imagine using these post hoc methods to drive discovery or precision medicine applications.…”

A review of connectivity map and computational approaches in pharmacogenomics

“…For data integration purposes, such platforms should remove biases of different sources such as batch effects, difference between profiling platforms and cell-specific differences to best characterize drug-induced effects. Furthermore, the unifying platforms should be easy to use so that users can develop new methods and functions for easy data manipulation and mining within the platform [105][106][107]. To address these issues, PharmacoGx, an open source package, has been recently developed [108].…”

Gene isoforms as expression-based biomarkers predictive of drug response in vitro

“…To address the lack of reproducibility of drug sensitivity measurements across studies [30,32], we developed a meta-analytical pipeline to combine the pharmacological data from CCLE and GDSC. The June 2014 release of CCLE consists of 11,670 experiments in which 24 drugs have been screened on 1,053 cancer cell lines from 24 tissue origins.…”

“…We recognize that using CCLE RNA-seq data in combination with GDSC is suboptimal as gene expression of cell lines are subject to biological and technical variations [33]. In the absence of RNA-seq data for GDSC, we could only address the variations observed in the drug sensitivity measurements, which we demonstrated to be significantly higher than variations in gene expression data [32]. To ensure that cell line identity was conserved across CCLE and GDSC, we performed SNP fingerprinting (Supplementary Methods) and filtered out the cell lines identified as different across studies using a cutoff of 80% concordance [32].…”

“…In the absence of RNA-seq data for GDSC, we could only address the variations observed in the drug sensitivity measurements, which we demonstrated to be significantly higher than variations in gene expression data [32]. To ensure that cell line identity was conserved across CCLE and GDSC, we performed SNP fingerprinting (Supplementary Methods) and filtered out the cell lines identified as different across studies using a cutoff of 80% concordance [32]. In addition we compared the microarray expression profiles of cell lines between microarray and RNA-seq profiles, which resulted in good concordance (Supplementary Figure 4) supporting that expression profiling are consistent.…”