XTQA: Span-Level Explanations of the Textbook Question Answering

Abstract

Textbook Question Answering (TQA) is a task that one should answer a diagram/nondiagram question given a large multi-modal context consisting of abundant essays and diagrams. We argue that the explainability of this task should place students as a key aspect to be considered. To address this issue, we devise a novel architecture towards span-level eXplanations of the TQA (XTQA). It can provide not only the answers but also the span-level evidences to choose them for students based on our proposed coarse-to-fine grained algorithm. The algorithm first coarsely chooses top M paragraphs relevant to questions using the TF-IDF method, and then chooses top K evidence spans finely from all candidate spans within these paragraphs by computing the information gain of each span to questions. Experimental results show that our method significantly improves the state-of-the-art performance compared with baselines. The source code is available at https://github.com/keepsmile-001/opentqa.

Weakly Supervised Learning for Textbook Question Answering

“…Explainability Practical TQA methods should not only answer textbook questions but provide students with explanations accurately, which helps them have a deeper understanding of what they have learned. There is only one work XTQA [11] researching on the TQA explainability. It regards the whole textual contexts of lessons as candidate evidence and applies a coarse-to-fine grained algorithm to extract spanlevel explanations for answering questions.…”

“…The questions can be classified into three categories including Non-Diagram True or False (NDTF) with two candidate answers, Non-Diagram Multiple-Choice (NDMC) with four to seven candidate answers, and Diagram-Multiple-Choice (DMC) with four candidate answers. Following previous works [11,14], we split TQA into NDTF, NDMC and DMC. We regard NDMC and DMC as a multi-class classification and consider NDTF as a binary classification.…”

“…In the following subsections, we do not use the subscripts to distinguish questions belonging to different subtasks, which may make readers easily understand our method. Following previous works [11,14], we devise a corresponding method for answering the questions of each subtask respectively.…”

“…Please Note AI2D only contains DMC questions. Following previous works [9,11,14], we use accuracy to evaluate our method. Implementation Details We introduce the implementation detail of each module within WSTQ as follows.…”

“…In this paper, we propose a Weakly Supervised learning method for TQA (WSTQ), in which the incompletely ac-curate results of essential intermediate procedures for TQA are regarded as supervision to develop Text Matching (TM) and Relation Detection (RD) tasks, and then the above tasks motivate the system to learn strong text comprehension and excellent diagram semantics respectively. Concretely, we apply the result of text retrieval that is an important intermediate procedure for TQA [11,12] and is used to find the most relevant text of questions, to develop TM. We consider the text t i , which is most relevant to the question q i in the lesson l u , as the matching text of q i and regard the text t j of q j ∈ l v , u = v, as the mismatching text of q i .…”