Most Random Gene Expression Signatures Are Significantly Associated with Breast Cancer Outcome

Abstract

Bridging the gap between animal or in vitro models and human disease is essential in medical research. Researchers often suggest that a biological mechanism is relevant to human cancer from the statistical association of a gene expression marker (a signature) of this mechanism, that was discovered in an experimental system, with disease outcome in humans. We examined this argument for breast cancer. Surprisingly, we found that gene expression signatures—unrelated to cancer—of the effect of postprandial laughter, of mice social defeat and of skin fibroblast localization were all significantly associated with breast cancer outcome. We next compared 47 published breast cancer outcome signatures to signatures made of random genes. Twenty-eight of them (60%) were not significantly better outcome predictors than random signatures of identical size and 11 (23%) were worst predictors than the median random signature. More than 90% of random signatures >100 genes were significant outcome predictors. We next derived a metagene, called meta-PCNA, by selecting the 1% genes most positively correlated with proliferation marker PCNA in a compendium of normal tissues expression. Adjusting breast cancer expression data for meta-PCNA abrogated almost entirely the outcome association of published and random signatures. We also found that, in the absence of adjustment, the hazard ratio of outcome association of a signature strongly correlated with meta-PCNA (R2 = 0.9). This relation also applied to single-gene expression markers. Moreover, >50% of the breast cancer transcriptome was correlated with meta-PCNA. A corollary was that purging cell cycle genes out of a signature failed to rule out the confounding effect of proliferation. Hence, it is questionable to suggest that a mechanism is relevant to human breast cancer from the finding that a gene expression marker for this mechanism predicts human breast cancer outcome, because most markers do. The methods we present help to overcome this problem.

Are expression profiles meaningless for cancer studies?

“…Indeed, several tests based on transcriptome analysis eventually achieved regulatory approval and are currently used in the clinic. However, a recent paper [8] strongly challenges the scientific validity of these profiles as applied to studies on breast (and other) cancers, and claims that 'Most published signatures are not significantly better outcome predictors than random signatures of identical size' 1 . In this commentary I address this paper and some of the relevant literature, showing that -even though the claim stated above is somewhat overstated -there is a significant component of truth in this conclusion, and important lessons can be gained from this work.…”

“…The results are presented in Fig. 2, an annotated version of the corresponding figure in the paper of Venet et al [8]. This shows the p value of the survival prediction for each of the published signatures applied to the NKI cohort (shown by a red dot) against the background of the predictions from 1,000 random signatures containing the same number of genes.…”

“…A: Survival curves for the two classes separated by the van de Vijver et al [11] signature. B: Survival curves for the two classes separated among the same patients by the 'social defeat' gene signature [8]. In both cases the fraction of surviving patients is given against elapsed time (in years).…”

“…In both cases the fraction of surviving patients is given against elapsed time (in years). Adapted from [11] (A) and from [8] …”

“…Even though statistical methods are much improved compared to initial work in this field [7], the design of experiments and the methods for error estimation still leave much to be desired, as discussed recently in this Journal [23]. Again, the analysis presented in the paper by Venet et al [8] does not negate the clinical worth of expression signatures. However, it does raise important questions about their mechanistic significance and the validity of many interpretations derived from them.…”