RNA secondary structure packages evaluated and improved by high-throughput experiments

Abstract

The computer-aided study and design of RNA molecules is increasingly prevalent across a range of disciplines, yet little is known about the accuracy of commonly used structure prediction packages in real-world tasks. Here, we evaluate the performance of current packages using EternaBench, a dataset comprising 23 in vitro structure mapping and 11 riboswitch activity datasets involving 18,509 synthetic sequences from the crowdsourced RNA design project Eterna. We find that CONTRAfold and RNAsoft, packages with parameters derived through statistical learning, achieve consistently higher accuracy than more widely used packages like the ViennaRNA software, which derive parameters primarily from thermodynamic experiments.Motivated by these results, we develop a multitask-learning-based model, EternaFold, which demonstrates improved performance that generalizes to diverse external datasets, including complete viral genomes probed in vivo and synthetic designs modeling mRNA vaccines.

Theoretical basis for stabilizing messenger RNA through secondary structure design

“…AUP solutions have CAI values of 0.91, 0.86, 0.61, and 0.73 respectively, which are comparable to vendor-generated sequences, suggesting that optimization of AUP will not result in a major penalty on CAI. Repeating the same analysis across other small model systems as well as with other secondary-structure packages ( 27 , 34 ) reveals similar results ( Figure S1 ).…”

“…Minimal AUP solutions have more stems and fewer 'hot spots' (7 vs. 15 yellow nucleotides in HA panel of Figures 2D vs. 2C) rather than optimize the folding free energy of each stem, once formed (reflected here in the base pairing probability; magenta vs. dark purple coloring in HA panel of Figures 2D vs. 2C). Repeating the sameanalysis across other small model systems as well as with other secondary-structure packages(27,34) reveals similar results (Figure S1). Taken together, this enumerative analysis of small model mRNAs suggested up to 2-fold increased stabilization might be achievable in mRNA design while retaining excellent codon adaptation indices and that solutions with minimal folding free energy are not necessarily expected to be most stable to hydrolysis.…”

“…Furthermore, the min.AUP solutions have CAI values of 0.91, 0.86, 0.61, and 0.73 respectively, which are comparable to vendor-generated sequences, suggesting that optimization of AUP will not result in a major penalty on CAI. Repeating the same analysis across other small model systems as well as with other secondarystructure packages(27,34) reveals similar results (Figure S1).How much can these small mRNAs be stabilized over standard mRNA design methods? We calculated the change in AUP obtained between the average of all the vendor-returned sequences and the minimum AUP solution for tag proteins, and found fold-changes in AUP ranging from 1.27 to 2.23fold decrease in AUP(Table 2).…”

ExpertRNA: A new framework for RNA structure prediction

“…Given the impact of structure on RNA functionality, the accurate computational prediction of the secondary and tertiary structure of RNA is an ongoing area of great interest in the computational biology community (Calonaci et al, 2020;Cruz et al, 2012;Wayment-Steele et al, 2020).…”

“…To the knowledge of the authors, ensemble methods allow to mix solutions from different algorithms only upon completion, i.e., they do not enable interaction among the algorithms while they are running (Aghaeepour and Hoos, 2013). A recent survey on a set of secondary structure prediction tools has reported mixed results, with data-driven approaches generally outperforming the ones based on nearest-neighbor free energy models, and with model-based ensemble approaches showing competitive results (Wayment-Steele et al, 2020).…”