How to benchmark RNA secondary structure prediction accuracy

Abstract

RNA secondary structure prediction is widely used. As new methods are developed, these are often benchmarked for accuracy against existing methods. This review discusses good practices for performing these benchmarks, including the choice of benchmarking structures, metrics to quantify accuracy, the importance of allowing flexibility for pairs in the accepted structure, and the importance of statistical testing for significance. Keywords Comparative Sequence Analysis; RNA Folding RNA structure is hierarchical, from primary, to secondary, to tertiary, and to quaternary structure [9]. The primary structure is the covalent structure, i.e. the sequence of nucleotides. The secondary structure is the set of canonical base pairs, including A-U, G-C, and G-U pairs. These base pairs are organized in A-form helices, which flank nucleotides that are said to be in loops. The tertiary structure is the three-dimensional positions of the atoms in space, which demonstrate the additional intramolecular contacts beyond canonical base pairing. These contacts include non-canonical base pairs, stacks, and base-backbone interactions.

Collaborative Learning of Estimation of Distribution Algorithm for RNA secondary structure prediction

“…The accuracy of RNA secondary structure prediction is found by comparing to a known structure [24]. Performance of all compared algorithms was measured using these metrics:…”

“…Zhang Kai,and Yulin Lv [14] proposed a multi-objective optimization algorithm including maximum base pair matching and minimum base-pair groups to evaluate the candidate solutions and adapt NSGA-II to find a group of non-dominated solutions. Despite the large body of research already published, developing improved methods for secondary structure prediction is a field of active research [24]. This paper proposes a novel collaborative learning algorithm, referred to as Co-EDAs, using two estimation of distribution algorithms for RNA secondary structure prediction.…”

Base-pair Ambiguity and the Kinetics of RNA Folding

“…This was demonstrated by statistical tests and, perhaps more meaningfully, by showing that the ambiguity index could be used to classify with good accuracy individual molecules as either bound or unbound. These experiments were based on comparative secondary structures available through the RNA STRAND database [19], which remains one of the most trusted sources for RNA secondary structures of single molecules [20][21][22].…”

“…We obtained comparative-analysis secondary structure data for seven different families of RNA molecules from the RNA STRAND database [19], a curated collection of RNA secondary structures which are widely used as reference structures for single RNA molecules [20][21][22]. These families include: Group I Introns and Group II Introns [43], tmRNAs and SRP RNAs [44], the Ribonuclease P RNAs [45], and 16s rRNAs and 23s rRNAs [43].…”

“…Recall now that the ambiguity index depends on both the primary and native secondary structures of the molecule, d = d(p, s), which raises the question-which secondary structure s should be used in the calculation? Our main conclusions were drawn using comparative secondary structures [17,18] available through the RNA STRAND database [19], a curated collection of RNA secondary structures which are widely used as reference structures for single RNA molecules [20][21][22]. [2] Native secondary structures often include so-called pseudoknots, which are sometimes excluded, or handled separately, for computational efficiency.…”