Robustifying Token Attention for Vision Transformers

Guo, Yong; Stutz, David; Schiele, Bernt

doi:10.1109/iccv51070.2023.01610

Cited by 11 publications

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

AMter: An end‐to‐end model for transcriptional terminators prediction by extracting semantic feature automatically based on attention mechanism

Zhang,

Li,

et al. 2024

Concurrency and Computation

View full text Add to dashboard Cite

SummaryThe Terminator, a specific DNA sequence, provides the transcriptional termination signal to RNA polymerase, making it a critical aspect of transcriptional regulation. This article proposes AMter, the first end‐to‐end model designed for predicting transcriptional terminators, leveraging attention mechanisms. In AMter, rather than manual feature engineering, two distinct modules based on attention mechanism, known as Frequency‐Attention and Allkmer‐Attention, are employed to automatically learn efficient features. Frequency‐Attention generates informative features by autonomously determining the significance of various frequency features, while Allkmer‐Attention aims to capture the relationships among all k‐mers within a DNA sequence. Features generated by Frequency‐Attention and Allkmer‐Attention demonstrate high informativeness and discriminative capacity, allowing precise discrimination of whether a DNA sequence is a terminator through a simple prediction network. The results of the 5‐fold cross‐validation test indicate the remarkable achievement of our proposed method, attaining 100% accuracy in both the training and validation datasets. Furthermore, AMter demonstrates outstanding prediction accuracy on two independent datasets, with 100% accuracy for Escherichia coli and 99.30% for Bacillus subtilis, marking a significant 94.4% relative improvement over prior methods. Experimental results conclusively demonstrate that AMter surpasses existing approaches, establishing a new state‐of‐the‐art in transcriptional terminator prediction.

show abstract