A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection

Bonadiman, Daniele; Moschitti, Alessandro

doi:10.18653/v1/2020.coling-main.457

Cited by 7 publications

(5 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Joint Model Multi-classifier performs lower than PR for all measures and all datasets. This is in line with the findings of Bonadiman and Moschitti (2020), who also did not obtain improvement when jointly used all the candidates altogether in a representation.…”

Section: Comparative Resultssupporting

confidence: 90%

“…The loss function for training such networks is constituted by the contribution of all elements of its ranked items. The closest work to our research is by Bonadiman and Moschitti (2020), who designed several joint models. These improved early neural networks based on CNN and LSTM for AS2, but failed to improve the state of the art using pre-trained Transformer models.…”

Section: Answer Sentence Selection (As2)mentioning

confidence: 99%

See 1 more Smart Citation

Joint Models for Answer Verification in Question Answering Systems

Zhang¹,

Vu²,

Moschitti³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

This paper studies joint models for selecting correct answer sentences among the top k provided by answer sentence selection (AS2) modules, which are core components of retrievalbased Question Answering (QA) systems. Our work shows that a critical step to effectively exploiting an answer set regards modeling the interrelated information between pair of answers. For this purpose, we build a three-way multiclassifier, which decides if an answer supports, refutes, or is neutral with respect to another one. More specifically, our neural architecture integrates a state-of-the-art AS2 module with the multi-classifier, and a joint layer connecting all components. We tested our models on Wik-iQA, TREC-QA, and a real-world dataset. The results show that our models obtain the new state of the art in AS2. * Work done while the author was an intern at Amazon Alexa Claim: Joe Walsh was inducted in 2001. Ev1: As a member of the Eagles, Walsh was inducted into the Rock and Roll Hall of Fame in 1998, and into the Vocal Group Hall of Fame in 2001. Ev2:Joseph Fidler Walsh (born November 20, 1947) is an American singer songwriter, composer, multiinstrumentalist and record producer. Ev3:Walsh was awarded with the Vocal Group Hall of Fame in 2001.

show abstract

Section: Comparative Resultssupporting

confidence: 90%

Section: Answer Sentence Selection (As2)mentioning

confidence: 99%

Joint Models for Answer Verification in Question Answering Systems

Zhang¹,

Vu²,

Moschitti³

2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

show abstract

“…Answer Sentence Selection TANDA (Garg et al, 2020) established the SOTA for AS2 using a large dataset (ASNQ) for transfer learning. Other approaches for AS2 include: separate encoders for question and answers (Bonadiman and Moschitti, 2020), and compare-aggregate and clustering to improve answer relevance ranking (Yoon et al, 2019). and SOP (Lan et al, 2020) have been widely explored for transformers to improve accuracy for downstream classification tasks.…”

Section: Related Workmentioning

confidence: 99%

Paragraph-based Transformer Pre-training for Multi-Sentence Inference

Liello¹,

Garg²,

Soldaini³

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Inference tasks such as answer sentence selection (AS2) or fact verification are typically solved by fine-tuning transformer-based models as individual sentence-pair classifiers. Recent studies show that these tasks benefit from modeling dependencies across multiple candidate sentences jointly. In this paper, we first show that popular pre-trained transformers perform poorly when used for fine-tuning on multi-candidate inference tasks. We then propose a new pre-training objective that models the paragraph-level semantics across multiple input sentences. Our evaluation on three AS2 and one fact verification datasets demonstrates the superiority of our pre-training technique over the traditional ones for transformers used as joint models for multi-candidate inference tasks, as well as when used as cross-encoders for sentence-pair formulations of these tasks.

show abstract

“…Then, Bonadiman and Moschitti (2020) attempted to design several joint models, which improved early neural models for AS2 but, when used in Transformer-based rerankers, they failed to improve the state of the art. Jin et al (2020) used the relation between candidates in Multi-task learning approach for AS2 but as they do not exploit transformer models, their results are rather lower than the state of the art.…”

Section: Related Workmentioning

confidence: 99%

Double Retrieval and Ranking for Accurate Question Answering

Zhang¹,

Vu²,

Moschitti³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Recent work hasshown that an answer verification step introduced in Transformer-based answer selection models can significantly improve the state of the art in Question Answering. This step is performed by aggregating the embeddings of top k answer candidates to support the verification of a target answer. Although the approach is intuitive and sound still shows two limitations: (i) the supporting candidates are ranked only according to the relevancy with the question and not with the answer, and (ii) the support provided by the other answer candidates is suboptimal as these are retrieved independently of the target answer. In this paper, we address both drawbacks by proposing (i) a double reranking model, which, for each target answer, selects the best support; and (ii) a second neural retrieval stage designed to encode question and answer pair as the query, which finds more specific verification information. The results on three wellknown datasets for AS2 show consistent and significant improvement of the state of the art.

show abstract

A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection

Cited by 7 publications

References 28 publications

Joint Models for Answer Verification in Question Answering Systems

Joint Models for Answer Verification in Question Answering Systems

Paragraph-based Transformer Pre-training for Multi-Sentence Inference

Double Retrieval and Ranking for Accurate Question Answering

Contact Info

Product

Resources

About