Enhancements of Attention-Based Bidirectional LSTM for Hybrid Automatic Text Summarization

Abstract

The automatic generation of a text summary is a task of generating a short summary for a relatively long text document by capturing its key information. In the past, supervised statistical machine learning was widely used for this Automatic Text Summarization (ATS) task, but due to its high dependence on the quality of text features, the generated summaries lack accuracy and coherence, while the computational power involved, and performance achieved, could not easily meet the current needs. This paper proposes four novel ATS models with a Sequence-to-Sequence (Seq2Seq) structure, utilizing an attention-based bidirectional Long Short-Term Memory (LSTM), with added enhancements for increasing the correlation between the generated text summary and the source text, and solving the problem of unregistered words, suppressing the repeated words, and preventing the spread of cumulative errors in generated text summaries. Experiments conducted on two public data sets confirmed that the proposed ATS models achieve indeed better performance than the baselines and some of the state-of-the-art models considered.

Two-Stage Intelligent Model for Detecting Malicious DDoS Behavior

“…After setting these hyperparameters, we trained the CNAT model proposed in the offline phase, deployed the trained model at the ingress router, and utilized the CNAT model to detect abnormal traffic in the online phase. Compared with the existing model [ 40 , 41 ], the CNAT model has a higher malicious traffic detection capability and a shorter response time. The experimental results show that the model can also detect 21 different types of DDoS attacks at the ingress router.…”

“…In order to verify our proposed model for DDoS attacks, we compared it with the LSAT model [ 40 ] and the CNLS model [ 41 ]. Then, the best deep learning model has been selected for the online detection of DDoS attacks.…”

“…With the development of neural networks, attention mechanisms are being proposed. Jiang et al [ 40 ] proposed four ATS models with a Sequence-to-Sequence model. An attention-based bidirectional LSTM (Long Short-Term Memory) can enhance the correlation between the generated text summary and the source text, which can prevent the spread of cumulative errors in generated text summaries.…”

“…We used 1DCNN to learn the packet header and payload features, utilized the attention mechanism to strengthen the neural network learning capability, and further fine-tuned the identification and classification. To verify our proposed model for DDoS attacks, we compared it with the LSAT model [ 40 ] and the LSCN model [ 41 ].…”

Automatic symptoms identification from a massive volume of unstructured medical consultations using deep neural and BERT models

“…Whilst, it holds the relevant information and controls its flow through the neural gates. The BiLSTM network showed efficient performance in handling sequences of data, which located it in an exceptional place for such tasks [10] . Thus and according to Faris et al in [11] , the BiLSTM is used for the identification of symptoms in combination with contextual features (embedding) engineered from the pre-training AraBERT model [12] .…”