Adversarial Training for Machine Reading Comprehension with Virtual Embeddings

Yang, Ziqing; Cui, Yiming; Si, Chenglei; Che, Wanxiang; Liu, Ting; Wang, Shijin; Hu, Guoping

doi:10.18653/v1/2021.starsem-1.30

Cited by 5 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They introduced two algorithms called AddSent and AddAny. Later, Yang et al (2021) improved these algorithms by introducing AddSentDivers to increase the diversity of the generated adversarial sentences.…”

Section: Adversarial Sentences In Qa Systemsmentioning

confidence: 99%

RobustQA: A Framework for Adversarial Text Generation Analysis on Question Answering Systems

Boreshban,

Mirbostani,

Ahmadi

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

View full text Add to dashboard Cite

Question answering (QA) systems have reached human-level accuracy; however, these systems are not robust enough and are vulnerable to adversarial examples. Recently, adversarial attacks have been widely investigated in text classification. However, there have been few research efforts on this topic in QA. In this article, we have modified the attack algorithms widely used in text classification to fit those algorithms for QA systems. We have evaluated the impact of various attack methods on QA systems at character, word, and sentence levels. Furthermore, we have developed a new framework, named RobustQA, as the first opensource toolkit for investigating textual adversarial attacks in QA systems. RobustQA consists of seven modules: Tokenizer, Victim Model, Goals, Metrics, Attacker, Attack Selector, and Evaluator. It currently supports six different attack algorithms. Furthermore, the framework simplifies the development of new attack algorithms in QA.

show abstract

Section: Adversarial Sentences In Qa Systemsmentioning

confidence: 99%

RobustQA: A Framework for Adversarial Text Generation Analysis on Question Answering Systems

Boreshban,

Mirbostani,

Ahmadi

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

View full text Add to dashboard Cite

show abstract

“…Wang and Jiang [11] combined general knowledge with neural networks through data augmentation. Yang et al [12,13] used adversarial training to maximize the countermeasure loss by adding perturbations in the embedding layer. In addition, some studies have attempted to change the process of model inference.…”

Section: Adversarial Attacks In Machine Reading Comprehension Modelmentioning

confidence: 99%

“…To further illustrate the advantages of our algorithm, we choose the following eleven methods for comparison: QAInfoMax [16], MAARS [9], R.M-Reader [27], KAR [11], BERT+Adv [12], ALUM [13], Sub-part Alignment [14], BERT+DGAdv [8], BERT+PR [15], HKAUP [28], and PQAT [13]. These eleven methods are used to improve the robustness of MRC model.…”

Section: Algorithm Comparisonmentioning

confidence: 99%

Improving the robustness of machine reading comprehension via contrastive learning

et al. 2022

View full text Add to dashboard Cite

Pre-trained language models achieve high performance on machine reading comprehension task, but these models lack robustness and are vulnerable to adversarial samples. Most of the current methods for improving model robustness are based on data enrichment. However, these methods do not solve the problem of poor context representation of the machine reading comprehension model. We find that context representation plays a key role in the robustness of the machine reading comprehension model, dense context representation space results in poor model robustness. To deal with this, we propose a Multi-task machine Reading Comprehension learning framework via Contrastive Learning. Its main idea is to improve the context representation space encoded by the machine reading comprehension models through contrastive learning. This special contrastive learning we proposed called Contrastive Learning in Context Representation Space(CLCRS). CLCRS samples sentences containing context information from the context as positive and negative samples, expanding the distance between the answer sentence and other sentences in the context. Therefore, the context representation space of the machine reading comprehension model has been expanded. The model can better distinguish between sentence containing correct answers and misleading sentence. Thus, the robustness of the model is improved. Experiment results on adversarial datasets show that our method exceeds the comparison models and achieves state-of-the-art performance.

show abstract

“…The performance of models also has been shown the improvement while applying Virtual Adversarial Training (author?) on SQuAD1.1 [12], SQuAD2.0 [13] and RACE [14]. According to the benefits of AT, we decided to apply several training strategies that can boost the model performance across MRC tasks which is discussed further in Section 2.3 and Section 2.4.…”

Section: Introductionmentioning

confidence: 99%

ViMRC - VLSP 2021: An empirical study of Vietnamese Machine Reading Comprehension with Unsupervised Context Selector and Adversarial Learning

Nguyen¹

2022

JCSCE

View full text Add to dashboard Cite

Machine Reading Comprehension (MRC) is a great NLP task that requires concentration on making the machine read, scan documents, and extract meaning from the text, just like a human reader.One of the MRC system challenges is not only having to understand the context to extract the answer but also being aware of the trust-worthy of the given question is possible or not.Thought pre-trained language models (PTMs) have shown their performance on many NLP downstream tasks, but it still has a limitation in the fixed-length input. We propose an unsupervised context selector that shortens the given context but still contains the answers within related contexts.In VLSP2021-MRC shared task dataset, we also empirical several training strategies consisting of unanswerable question sample selection and different adversarial training approaches, which slightly boost the performance 2.5% in EM score and 1% in F1 score.

show abstract

Adversarial Training for Machine Reading Comprehension with Virtual Embeddings

Cited by 5 publications

References 16 publications

RobustQA: A Framework for Adversarial Text Generation Analysis on Question Answering Systems

RobustQA: A Framework for Adversarial Text Generation Analysis on Question Answering Systems

Improving the robustness of machine reading comprehension via contrastive learning

ViMRC - VLSP 2021: An empirical study of Vietnamese Machine Reading Comprehension with Unsupervised Context Selector and Adversarial Learning

Contact Info

Product

Resources

About