Enhancer-LSTMAtt: A Bi-LSTM and Attention-Based Deep Learning Method for Enhancer Recognition

Abstract

Enhancers are short DNA segments that play a key role in biological processes, such as accelerating transcription of target genes. Since the enhancer resides anywhere in a genome sequence, it is difficult to precisely identify enhancers. We presented a bi-directional long-short term memory (Bi-LSTM) and attention-based deep learning method (Enhancer-LSTMAtt) for enhancer recognition. Enhancer-LSTMAtt is an end-to-end deep learning model that consists mainly of deep residual neural network, Bi-LSTM, and feed-forward attention. We extensively compared the Enhancer-LSTMAtt with 19 state-of-the-art methods by 5-fold cross validation, 10-fold cross validation and independent test. Enhancer-LSTMAtt achieved competitive performances, especially in the independent test. We realized Enhancer-LSTMAtt into a user-friendly web application. Enhancer-LSTMAtt is applicable not only to recognizing enhancers, but also to distinguishing strong enhancer from weak enhancers. Enhancer-LSTMAtt is believed to become a promising tool for identifying enhancers.

A comprehensive revisit of the machine‐learning tools developed for the identification of enhancers in the human genome

“…Eight different DL‐based methods used different architectures with feature encodings: (i) Enhancer‐LSTMAtt [70] was an end‐to‐end DL model consisting of a Bi‐LSTM, deep residual network, and feed‐forward attention. It is trained using the word2vec feature.…”

“…In stage 3, a variety of ML-based and DL-based approaches were used, including support vector machine (SVM) [52][53][54][55]89], extreme gradient boosting (XGB) [59,90], gradient-boosting decision trees (GB) [91], random forest (RF) [58,[92][93][94], AdaBoost, extremely randomized tree (ERT) [72], deep forest [95], light gradient boosting machine (LGB) [73], naive Bayesian (NB) [73], k-nearest neighbor (KNN) [73], multilayer perceptron (MLP) [72], DNN [66,96], CNN [63,67,97], residual CNN (RCNN) [98], RNN [99,100], gated recurrent unit (GRU)-based bidirectional RNN (BRNN) [101], deep belief network (DBN), Bi-LSTM [70,86], feed-forward attention [70], residual network (ResNet), and RBFN [102]. In some methods, the predicted values of different classifiers are incorporated and input into specific classifiers to develop the final prediction model.…”

“…Conventional ML classifier‐based methods, including iEnhancer‐2L [52], EnhancerPred [53], iEnhancer‐PsedeKNC [54], EnhancerPred2.0 [55], iEnhancer‐EL [56], iEnhancer‐2L‐Hybrid [57], iEnhancer‐RF [58], iEnhancer‐XG [59], piEnPred [60], and iEnhancer‐KL [61]. DL‐based methods, including DeepEnhancer [62], iEnhancer‐ECNN [63], iEnhancer‐CNN [64], RankGAN [65], Enhancer‐DSNet [66], BERT‐Enhancer [67], iEnhancer‐DHF [68], iEnhancer‐EBLSTM [69], Enhancer‐LSTMAtt [70], and iEnhancer‐BERT [71]. Hybrid methods, including Enhancer‐IF [72] and Enhancer‐FRL [73].…”

“…[69], Enhancer-LSTMAtt [70], and iEnhancer-BERT [71]. Hybrid methods, including Enhancer-IF [72] and Enhancer-FRL [73].…”

“…Enhancer tools developed in 2022DL-based methods have been published in greater numbers than conventional ML-based methods (9 vs. 2) in 2022. These methods were implemented using Liu dataset1 or Liu dataset2.Eight different DL-based methods used different architectures with feature encodings: (i) Enhancer-LSTMAtt[70] was an end-to-end DL model consisting of a Bi-LSTM, deep residual network, and feedforward attention. It is trained using the word2vec feature.…”