Predicting and Integrating Expected Answer Types into a Simple Recurrent Neural Network Model for Answer Sentence Selection

Kamath, Sanjay; Grau, Brigitte; Ma, Yue

doi:10.13053/cys-23-3-3241

Cited by 13 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e impressive effectiveness of this technique was confirmed by applying it to the model presented in [73]. Meanwhile, the authors in [75] claimed that not all the named entities could be replaced with one token, so they considered a token for each named entity. It was later found that using the attention mechanism could produce more valuable models.…”

Section: Related Workmentioning

confidence: 92%

See 1 more Smart Citation

RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm

Gharagozlou

Mohammadzadeh

Bastanfard

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Answer selection (AS) is a critical subtask of the open-domain question answering (QA) problem. The present paper proposes a method called RLAS-BIABC for AS, which is established on attention mechanism-based long short-term memory (LSTM) and the bidirectional encoder representations from transformers (BERT) word embedding, enriched by an improved artificial bee colony (ABC) algorithm for pretraining and a reinforcement learning-based algorithm for training backpropagation (BP) algorithm. BERT can be comprised in downstream work and fine-tuned as a united task-specific architecture, and the pretrained BERT model can grab different linguistic effects. Existing algorithms typically train the AS model with positive-negative pairs for a two-class classifier. A positive pair contains a question and a genuine answer, while a negative one includes a question and a fake answer. The output should be one for positive and zero for negative pairs. Typically, negative pairs are more than positive, leading to an imbalanced classification that drastically reduces system performance. To deal with it, we define classification as a sequential decision-making process in which the agent takes a sample at each step and classifies it. For each classification operation, the agent receives a reward, in which the prize of the majority class is less than the reward of the minority class. Ultimately, the agent finds the optimal value for the policy weights. We initialize the policy weights with the improved ABC algorithm. The initial value technique can prevent problems such as getting stuck in the local optimum. Although ABC serves well in most tasks, there is still a weakness in the ABC algorithm that disregards the fitness of related pairs of individuals in discovering a neighboring food source position. Therefore, this paper also proposes a mutual learning technique that modifies the produced candidate food source with the higher fitness between two individuals selected by a mutual learning factor. We tested our model on three datasets, LegalQA, TrecQA, and WikiQA, and the results show that RLAS-BIABC can be recognized as a state-of-the-art method.

show abstract

Section: Related Workmentioning

confidence: 92%

“…is method uses knowledge graphs (KG) to learn the representation of questions and answers. EATS [75] adopted an RNN network to measure the similarity between the QA pair. First, it replaces each named entity with a specific word.…”

Section: Baseline Methodsmentioning

confidence: 99%

RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm

Gharagozlou

Mohammadzadeh

Bastanfard

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…This component extracts EAT from the question. EAT shows the type of the answers to the questions [ 35 ]. For example, the EATs for the questions who is the best soccer player in history?…”

Section: The Proposed Methodsmentioning

confidence: 99%

“…The authors believed that using pairwise ranking rather than using pointwise ranking leads to the generation of high-quality output vector representations for the question and the candidate answer. Kamath et al [ 35 ] used a simple recurrent neural network (RNN) as shared-weight neural network and employed logistic regression to calculated the similarity between the question and the candidate answer. However, they showed that integrating question classification and answer selection component eliminates the requirement of a heavy-weight neural network to solve the answer selection task.…”

Section: Related Workmentioning

confidence: 99%

PerAnSel: A Novel Deep Neural Network-Based System for Persian Question Answering

Mozafari

Kazemi

Moradi

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Question answering (QA) systems have attracted considerable attention in recent years. They receive the user’s questions in natural language and respond to them with precise answers. Most of the works on QA were initially proposed for the English language, but some research studies have recently been performed on non-English languages. Answer selection (AS) is a critical component in QA systems. To the best of our knowledge, there is no research on AS for the Persian language. Persian is a (1) free word order, (2) right-to-left, (3) morphologically rich, and (4) low-resource language. Deep learning (DL) techniques have shown promising accuracy in AS. Although DL performs very well on QA, it requires a considerable amount of annotated data for training. Many annotated datasets have been built for the AS task; most of them are exclusively in English. In order to address the need for a high-quality AS dataset in the Persian language, we present PASD; the first large-scale native AS dataset for the Persian language. To show the quality of PASD, we employed it to train state-of-the-art QA systems. We also present PerAnSel: a novel deep neural network-based system for Persian question answering. Since the Persian language is a free word-order language, in PerAnSel, we parallelize a sequential method and a transformer-based method to handle various orders in the Persian language. We then evaluate PerAnSel on three datasets: PASD, PerCQA, and WikiFA. The experimental results indicate strong performance on the Persian datasets beating state-of-the-art answer selection methods by 10.66 % on PASD, 8.42 % on PerCQA, and 3.08 % on WikiFA datasets in terms of MRR.

show abstract

“…By using such a model on BIOASQ dataset which is a small scale labelled dataset, it will not result in similar performance as on the large scale open domain datasets due to overfitting. One way of overcoming this problem as reported by [6,15] is by pre-training a deep learning model on a large scale dataset and fine-tuning the same model to the target small scale dataset. The intuition is that the model learns better representations when learnt on a large scale dataset than having a randomly initialized model trained only on the small scale dataset.…”

Section: Introductionmentioning

confidence: 99%

How to Pre-train Your Model? Comparison of Different Pre-training Models for Biomedical Question Answering

Kamath

Grau

2020

Machine Learning and Knowledge Discovery in Databases

Self Cite

View full text Add to dashboard Cite

Using deep learning models on small scale datasets would result in overfitting. To overcome this problem, the process of pre-training a model and fine-tuning it to the small scale dataset has been used extensively in domains such as image processing. Similarly for question answering, pre-training and finetuning can be done in several ways. Commonly reading comprehension models are used for pre-training, but we show that other types of pre-training can work better. We compare two pre-training models based on reading comprehension and open domain question answering models and determine the performance when fine-tuned and tested over BIOASQ question answering dataset. We find open domain question answering model to be a better fit for this task rather than reading comprehension model.

show abstract

Predicting and Integrating Expected Answer Types into a Simple Recurrent Neural Network Model for Answer Sentence Selection

Cited by 13 publications

References 14 publications

RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm

RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm

PerAnSel: A Novel Deep Neural Network-Based System for Persian Question Answering

How to Pre-train Your Model? Comparison of Different Pre-training Models for Biomedical Question Answering

Contact Info

Product

Resources

About