Predicting RNA solvent accessibility from multi-scale context feature via multi-shot neural network

Fan, Xueqiang; Hu, Jun; Tang, Yu-Xuan; Jia, Ning-Xin; Yu, Dong‐Jun; Zhang, Guijun

doi:10.1016/j.ab.2022.114802

Cited by 3 publications

(3 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After this work was completed, we noticed a new method called M 2 pred published on RNA solvent accessibility prediction, which utilized multi-scale context features via a multi-shot neural network ( 66 ). We downloaded this evolution profile-based method from https://github.com/XueQiangFan/M2pred/ and compared it with RNA-MSM.…”

Section: Discussionmentioning

confidence: 99%

Multiple sequence alignment-based RNA language model and its application to structural inference

Zhang,

Lang,

Jiang

et al. 2023

Nucleic Acids Research

View full text Add to dashboard Cite

Compared with proteins, DNA and RNA are more difficult languages to interpret because four-letter coded DNA/RNA sequences have less information content than 20-letter coded protein sequences. While BERT (Bidirectional Encoder Representations from Transformers)-like language models have been developed for RNA, they are ineffective at capturing the evolutionary information from homologous sequences because unlike proteins, RNA sequences are less conserved. Here, we have developed an unsupervised multiple sequence alignment-based RNA language model (RNA-MSM) by utilizing homologous sequences from an automatic pipeline, RNAcmap, as it can provide significantly more homologous sequences than manually annotated Rfam. We demonstrate that the resulting unsupervised, two-dimensional attention maps and one-dimensional embeddings from RNA-MSM contain structural information. In fact, they can be directly mapped with high accuracy to 2D base pairing probabilities and 1D solvent accessibilities, respectively. Further fine-tuning led to significantly improved performance on these two downstream tasks compared with existing state-of-the-art techniques including SPOT-RNA2 and RNAsnap2. By comparison, RNA-FM, a BERT-based RNA language model, performs worse than one-hot encoding with its embedding in base pair and solvent-accessible surface area prediction. We anticipate that the pre-trained RNA-MSM model can be fine-tuned on many other tasks related to RNA structure and function.

show abstract

Section: Discussionmentioning

confidence: 99%

Multiple sequence alignment-based RNA language model and its application to structural inference

Zhang,

Lang,

Jiang

et al. 2023

Nucleic Acids Research

View full text Add to dashboard Cite

show abstract

“…After this work was completed, we noticed a new method called M 2 pred published on RNA solvent accessibility prediction, which utilized multi-scale context features via multi-shot neural network 55 . We downloaded this evolution-profile-based method from https://github.com/XueQiangFan/M2pred/ and compared it with RNA-MSM.…”

Section: Discussionmentioning

confidence: 99%

Multiple sequence-alignment-based RNA language model and its application to structural inference

Zhang

Lang

Jiang

et al. 2023

Preprint

View full text Add to dashboard Cite

Compared to proteins, DNA and RNA are more difficult languages to interpret because 4-letter-coded DNA/RNA sequences have less information content than 20-letter-coded protein sequences. While BERT (Bidirectional Encoder Representations from Transformers)-like language models have been developed for RNA, they are ineffective at capturing the evolutionary information from homologous sequences because unlike proteins, RNA sequences are less conserved. Here, we have developed an unsupervised Multiple sequence-alignment-based RNA language model (RNA-MSM) by utilizing homologous sequences from an automatic pipeline, RNAcmap. The resulting unsupervised, two-dimensional attention maps and one-dimensional embeddings from RNA-MSM can be directly mapped with high accuracy to 2D base pairing probabilities and 1D solvent accessibilities, respectively. Further fine-tuning led to significantly improved performance on these two downstream tasks over existing state-of-the-art techniques. We anticipate that the pre-trained RNA-MSM model can be fine-tuned on many other tasks related to RNA structure and function.

show abstract

“…The model employed modified attention layers with different receptive fields to conform to the stem-loop structure of RNA chains. Fan et al [62] proposed a novel computational method called M(2)pred for accurately predicting the solvent accessibility of RNA. The model utilized a multi-shot neural network with a multi-scale context feature extraction strategy.…”

Section: Sequence and Site Predictionmentioning

confidence: 99%

Transformer Architecture and Attention Mechanisms in Genome Data Analysis: A Comprehensive Review

Choi

Lee

2023

Biology

View full text Add to dashboard Cite

The emergence and rapid development of deep learning, specifically transformer-based architectures and attention mechanisms, have had transformative implications across several domains, including bioinformatics and genome data analysis. The analogous nature of genome sequences to language texts has enabled the application of techniques that have exhibited success in fields ranging from natural language processing to genomic data. This review provides a comprehensive analysis of the most recent advancements in the application of transformer architectures and attention mechanisms to genome and transcriptome data. The focus of this review is on the critical evaluation of these techniques, discussing their advantages and limitations in the context of genome data analysis. With the swift pace of development in deep learning methodologies, it becomes vital to continually assess and reflect on the current standing and future direction of the research. Therefore, this review aims to serve as a timely resource for both seasoned researchers and newcomers, offering a panoramic view of the recent advancements and elucidating the state-of-the-art applications in the field. Furthermore, this review paper serves to highlight potential areas of future investigation by critically evaluating studies from 2019 to 2023, thereby acting as a stepping-stone for further research endeavors.

show abstract

Predicting RNA solvent accessibility from multi-scale context feature via multi-shot neural network

Cited by 3 publications

References 44 publications

Multiple sequence alignment-based RNA language model and its application to structural inference

Multiple sequence alignment-based RNA language model and its application to structural inference

Multiple sequence-alignment-based RNA language model and its application to structural inference

Transformer Architecture and Attention Mechanisms in Genome Data Analysis: A Comprehensive Review

Contact Info

Product

Resources

About