A Stacked BiLSTM Neural Network Based on Coattention Mechanism for Question Answering

Cai, Linqin; Sitong, Zhou; Yan, Xun; Ren, Yuan

doi:10.1155/2019/9543490

Cited by 41 publications

(20 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Bidirectional LSTM is an extension of the traditional LSTM and is able to utilize related information from both the previous and future context. In the next section, we exploit the BiLSTM model for depression classification on a given input text (Cai et al 2019).…”

Section: Proposed Methodologymentioning

confidence: 99%

Applying Deep Learning Technique for Depression Classification in Social Media Text

Ahmad¹,

Asghar²,

Alotaibi³

et al. 2020

j med imaging hlth inform

View full text Add to dashboard Cite

In social media, depression identification could be regarded as a complex task because of the complicated nature associated with mental disorders. In recent times, there has been an evolution in this research area with growing popularity of social media platforms as these have become a fundamental part of people's day-to-day life. Social media platforms and their users share a close relationship due to which the users' personal life is reflected in these platforms on several levels. Apart from the associated complexity in recognising mental illnesses via social media platforms, implementing supervised machine learning approaches like deep neural networks is yet to be adopted in a large scale because of the inherent difficulties associated with procuring sufficient quantities of annotated training data. Because of such reasons, we have made effort to identify deep learning model that is most effective from amongst selected architectures with previous successful record in supervised learning methods. The selected model is employed to recognise online users that display depression; since there is limited unstructured text data that could be extracted from Twitter.

show abstract

Section: Proposed Methodologymentioning

confidence: 99%

Applying Deep Learning Technique for Depression Classification in Social Media Text

Ahmad¹,

Asghar²,

Alotaibi³

et al. 2020

j med imaging hlth inform

View full text Add to dashboard Cite

show abstract

“…This study used different conventional classifiers for CVP, including Adaboost [25], support vector machine (SVM) [26], logistic regression (LR) [27], random forest (RF) [28,29], Gaussian naïve Bayes (GNB) [30], decision tree C4.5 [31], and classification and regression trees (CART) [32]. For NVP, since the input is a variable-length sequence, we used LSTM [33], bidirectional LSTM (biLSTM) [34], Stack-LSTM [35], Stack-biLSTM [36], and Attention-LSTM [37].…”

Section: Common Classifiers Used In This Studymentioning

confidence: 99%

Current-Visit and Next-Visit Prediction for Fatty Liver Disease With a Large-Scale Dataset: Model Development and Performance Comparison

Chu

Jang

2021

JMIR Med Inform

View full text Add to dashboard Cite

Background Fatty liver disease (FLD) arises from the accumulation of fat in the liver and may cause liver inflammation, which, if not well controlled, may develop into liver fibrosis, cirrhosis, or even hepatocellular carcinoma. Objective We describe the construction of machine-learning models for current-visit prediction (CVP), which can help physicians obtain more information for accurate diagnosis, and next-visit prediction (NVP), which can help physicians provide potential high-risk patients with advice to effectively prevent FLD. Methods The large-scale and high-dimensional dataset used in this study comes from Taipei MJ Health Research Foundation in Taiwan. We used one-pass ranking and sequential forward selection (SFS) for feature selection in FLD prediction. For CVP, we explored multiple models, including k-nearest-neighbor classifier (KNNC), Adaboost, support vector machine (SVM), logistic regression (LR), random forest (RF), Gaussian naïve Bayes (GNB), decision trees C4.5 (C4.5), and classification and regression trees (CART). For NVP, we used long short-term memory (LSTM) and several of its variants as sequence classifiers that use various input sets for prediction. Model performance was evaluated based on two criteria: the accuracy of the test set and the intersection over union/coverage between the features selected by one-pass ranking/SFS and by domain experts. The accuracy, precision, recall, F-measure, and area under the receiver operating characteristic curve were calculated for both CVP and NVP for males and females, respectively. Results After data cleaning, the dataset included 34,856 and 31,394 unique visits respectively for males and females for the period 2009-2016. The test accuracy of CVP using KNNC, Adaboost, SVM, LR, RF, GNB, C4.5, and CART was respectively 84.28%, 83.84%, 82.22%, 82.21%, 76.03%, 75.78%, and 75.53%. The test accuracy of NVP using LSTM, bidirectional LSTM (biLSTM), Stack-LSTM, Stack-biLSTM, and Attention-LSTM was respectively 76.54%, 76.66%, 77.23%, 76.84%, and 77.31% for fixed-interval features, and was 79.29%, 79.12%, 79.32%, 79.29%, and 78.36%, respectively, for variable-interval features. Conclusions This study explored a large-scale FLD dataset with high dimensionality. We developed FLD prediction models for CVP and NVP. We also implemented efficient feature selection schemes for current- and next-visit prediction to compare the automatically selected features with expert-selected features. In particular, NVP emerged as more valuable from the viewpoint of preventive medicine. For NVP, we propose use of feature set 2 (with variable intervals), which is more compact and flexible. We have also tested several variants of LSTM in combination with two feature sets to identify the best match for male and female FLD prediction. More specifically, the best model for males was Stack-LSTM using feature set 2 (with 79.32% accuracy), whereas the best model for females was LSTM using feature set 1 (with 81.90% accuracy).

show abstract

“…where W and b represent the weight vectors and bias terms corresponding to the three gates of the LSTM, respectively. In relevant experiments [43]- [46], reasonable stacking of the networks can effectively improve model's ability of classification and regression. Therefore, this paper construct a stacked BiLSTM network based on the LSTM cell to fully achieve semantic parsing of a single statement.…”

Section: A Semantic Analysis With the Fusion Structure Of Cnn And Bimentioning

confidence: 99%

Intelligent Question Answering in Restricted Domains Using Deep Learning and Question Pair Matching

Cai¹,

Min²,

Sitong³

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

With the rapid expansion of the Internet, intelligent question answering for information retrieval has once again gained widespread attention. However, current question answering models mainly focus on the general and common-sense questions in open domains and are incapable to effectively solve more complex professional domain questions. This paper proposed an integrated framework for Chinese intelligent question answering in restricted domains. The proposed model fused convolutional neural network and bidirectional long short-term memory network which performs efficient semantic analysis on the question pairs to extract more effective features of the text. Meanwhile, the coattention mechanism and attention mechanism were combined to obtain the semantic interaction and feature representation of the question pair for providing complete information for subsequent calculations. In addition, we introduced the method of question pair matching to implement the Chinese intelligent question answering in a restricted domain. Experiments were tested and evaluated on the open-source CCKS2018 dataset and our private self-built inverted pendulum control question answering (IPC-QA) dataset for automation control virtual learning environment. Experimental results confirm that the proposed models are efficient and achieve a high precision of 0.86042 and 0.8031 on CCKS2018 and IPC-QA respectively.

show abstract

A Stacked BiLSTM Neural Network Based on Coattention Mechanism for Question Answering

Cited by 41 publications

References 32 publications

Applying Deep Learning Technique for Depression Classification in Social Media Text

Applying Deep Learning Technique for Depression Classification in Social Media Text

Current-Visit and Next-Visit Prediction for Fatty Liver Disease With a Large-Scale Dataset: Model Development and Performance Comparison

Intelligent Question Answering in Restricted Domains Using Deep Learning and Question Pair Matching

Contact Info

Product

Resources

About