Accuracy of mutational signature software on correlated signatures

Abstract

Mutational signatures are characteristic patterns of mutations generated by exogenous mutagens or by endogenous mutational processes. Mutational signatures are important for research into DNA damage and repair, aging, cancer biology, genetic toxicology, and epidemiology. Unsupervised learning can infer mutational signatures from the somatic mutations in large numbers of tumors, and separating correlated signatures is a notable challenge for this task. To investigate which methods can best meet this challenge, we assessed 18 computational methods for inferring mutational signatures on 20 synthetic data sets that incorporated varying degrees of correlated activity of two common mutational signatures. Performance varied widely, and four methods noticeably outperformed the others: hdp (based on hierarchical Dirichlet processes), SigProExtractor (based on multiple non-negative matrix factorizations over resampled data), TCSM (based on an approach used in document topic analysis), and mutSpec.NMF (also based on non-negative matrix factorization). The results underscored the complexities of mutational signature extraction, including the importance and difficulty of determining the correct number of signatures and the importance of hyperparameters. Our findings indicate directions for improvement of the software and show a need for care when interpreting results from any of these methods, including the need for assessing sensitivity of the results to input parameters.

Mutational signatures association with replication timing in normal cells reveals similarities and differences with matched cancer tissues

“…First, it is worth mentioning that differences in occurrences of signatures in ERR vs. LRR can be affected by the sensitivity of the mutational signature’s extraction method, missing low magnitude exposure, and not solely due to regional differences 15 . However, we used one of the most accurate, robust and well-known frameworks 34 , and the results are consistent across cancer types and studies. Moreover, we performed the analyses using an additional mutational signatures algorithm, and found similar results (Supplementary Fig.…”

Cancer mutational signatures identification in clinical assays using neural embedding-based representations

“…As with any signature algorithm, the SigProfiler-based annotations 2 are not flawless. 30 , 31 We found that some of MESiCA’s “mistakes” were probably due to miss-annotation. For example, 31/33 (93.9%) of POLE samples which MESiCA “failed” to predict were probably falsely assigned to SBS10a-b, as their mutational load was not high as expected in POLE-mutated samples, their tumor types were not typical of POLE signatures, and no hotspot mutations in POLE were detected ( Figure S3 A).…”

mSigHdp: hierarchical Dirichlet process mixture modeling for mutational signature discovery

“…While there has been some systematic benchmarking of mutational signature discovery methods on SBS data (Islam et al, 2020; Omichessan, Severi, & Perduca, 2019; Wu, Chua, Ng, Boot, & Rozen, 2022), we are unaware of previous benchmarking efforts on indel data. Based on analysis of reconstruction errors, we believe that the amount of resampling noise in the synthetic SBS and indel mutation data sets used here more closely resembles variability in real data than some previously used benchmarking data (Supplementary Figures S1,S2).…”

“…We assessed the ability of mSigHdp with and without downsampling to discover mutational signatures in synthetic SBS and indel mutation data, and we compared this to the abilities of 4 other programs. While there has been some systematic benchmarking of mutational signature discovery methods on SBS data (1,6,(22)(23)(24) we are unaware of previous benchmarking efforts on indel data. Based on an analysis of the reconstruction of mutational spectra in real tumors using known signatures, we believe that the amount of resampling noise in the synthetic SBS and indel mutation data sets used here more closely resembles variability in real data than some previously used benchmarking data (Supplementary Figures S1, S2).…”