Question Answering over Electronic Devices: A New Benchmark Dataset and a Multi-Task Learning based QA Framework

Abstract

Answering questions asked from instructional corpora such as E-manuals, recipe books, etc., has been far less studied than open-domain factoid context-based question answering. This can be primarily attributed to the absence of standard benchmark datasets. In this paper we meticulously create a large amount of data connected with E-manuals and develop suitable algorithm to exploit it. We collect E-Manual Corpus, a huge corpus of 307,957 E-manuals and pretrain RoBERTa on this large corpus. We create various benchmark QA datasets which include question answer pairs curated by experts based upon two E-manuals, real user questions from Community Question Answering Forum pertaining to E-manuals etc.We introduce EMQAP (E-Manual Question Answering Pipeline) that answers questions pertaining to electronics devices. Built upon the pretrained RoBERTa, it harbors a supervised multi-task learning framework which efficiently performs the dual tasks of identifying the section in the E-manual where the answer can be found and the exact answer span within that section.For E-Manual annotated question-answer pairs, we show an improvement of about 40% in ROUGE-L F1 scores over the most competitive baseline. We perform a detailed ablation study and establish the versatility of EMQAP across different circumstances. The code and datasets are shared at https://github.com/abhi1nandy2/ EMNLP-2021-Findings, and the corresponding project website is https://sites. google.com/view/emanualqa/home.

Commercialization of state-owned broadcast networks in China

“…Different from previous multimodal inputs, the product manual is a specific domain in terms of the question type and the content. Since product manuals usually contain detailed operation instructions for a specific device, the questions beginning with 'How to' are very common (Nandy et al 2021), while this type of contents and questions rarely occur in general domain datasets. Moreover, the answers in the abovementioned works are all in text format, including text span, multi-choice, and generative sentences.…”

“…The product manuals in PM209 are from two sources: 1) E-manual corpus (Nandy et al 2021); 2) official websites of the brands.…”

“…Source 1: E-manual corpus E-manual corpus (Nandy et al 2021) is a large-scale text corpus. It is constructed by crawling product manuals from the website 2 and extracting their text contents.…”

“…Each question is associated with a multimodal answer which is comprised of two parts: a textual part in natural language sentences, and a visual part containing regions from the manual. Table 1 shows the basic comparison between PM209 and existing PMQA datasets (Nandy et al 2021). The scale of PM209 is larger than existing PMQA datasets in terms of brands, manual numbers, and QA pairs.…”

“…Table8shows that PM209 covers diverse topics including PC, automobiles, speakers, applications, hardware, cellphones, cameras, etc. The topics of PM209 are much richer than S10 QA and Smart TV/Remote QA(Nandy et al 2021) which only focus on one specific product. Topics in PM209 are also very different from VisualMRC(Tanaka, Nishida, and Yoshida 2021) which consists of open-domain webpages and contains rare topics about specific consumer products.…”