Assessment of blind predictions of the clinical significance of BRCA1 and BRCA2 variants

Abstract

Testing for variation in BRCA1 and BRCA2 (commonly referred to as BRCA1/2), has emerged as a standard clinical practice and is helping countless women better understand and manage their heritable risk of breast and ovarian cancer. Yet the increased rate of BRCA1/2 testing has led to an increasing number of Variants of Uncertain Significance (VUS), and the rate of VUS discovery currently outpaces the rate of clinical variant interpretation. Computational prediction is a key component of the variant interpretation pipeline. In the CAGI5 ENIGMA Challenge, six prediction teams submitted predictions on 326 newly‐interpreted variants from the ENIGMA Consortium. By evaluating these predictions against the new interpretations, we have gained a number of insights on the state of the art of variant prediction and specific steps to further advance this state of the art.

Large scale multifactorial likelihood quantitative analysis of BRCA1 and BRCA2 variants: An ENIGMA resource to support clinical variant classification

“…Following this line of thought, it will be important that estimates of prior probability based on bioinformatic predictions are re‐estimated using updated datasets that reflect changed variant pools, and altered patient ascertainment in the era of multigene panel testing. Results arising from the CAGI 5 experiment (Cline et al, ), and other similar studies, are likely to inform development of such bioinformatic methods.…”

“…A prepublication iteration of the classification dataset was used as a reference set for the Critical Assessment of Genome Interpretation (CAGI) 5 experiment, results from which are presented in this same Journal issue (Cline et al, ). Comparison of various different prediction methods from six different teams highlighted that prediction of mRNA splicing is an important inclusion in variant interpretation algorithms, and also indicated that variant interpretation may be improved by incorporating amino acid accessibility as a component of bioinformatic prediction of variant effect.…”

Predicting pathogenicity of missense variants with weakly supervised regression

“…We next compare three types of machine learning models, as previously summarized in Table , for the task of pathogenicity prediction: MLR , a representative of multiclass classification used by other participants in the Challenge (Cline et al, ); CLM , a representative of multiclass classification that considers the order among classes (ordinal regression); and our three WSR models that directly predict the probability of pathogenicity from training pathogenicity classes using designed loss functions, summarized and compared in Table . Due to the lack of ground‐truth PoP values, we were only able to assess classification performances.…”

“…Indeed, the mutation was not found to destabilize BRCA1 (Figure S2). Interestingly, the top‐performing LEAP team found that splicing information, such as the distance to the nearest splice site, was also important for correctly annotating the clinical significance of some variants (Cline et al, ).…”

“…We describe three types of machine learning models. The first two do not predict PoP ( p i ) directly but classify pathogenicity ( y i ) instead: Multiclass logistic regression (MLR), a representative multiclass classification method used by other participants in the Challenge (Cline et al, ), disregard the order among the five classes; and cumulative logit model (CLM; Agresti, ), a representative of ordinal regression, treat the classes as ordered albeit on an arbitrary scale. In contrast, the last, including three “weakly supervised” regressors (WSR) developed by us, directly predict PoP ( p i ) for variant i by training models on inexact labels (not PoP but PoP ranges encoded by pathogenicity class y i ) while utilizing both the order and the scale among the classes.…”