Sim2RealQA: Using Life Simulation to Solve Question Answering Real-World Events

Abstract

As smart speakers continue to proliferate, question answering (QA) by smart devices is being woven into our daily lives. This study assumes question answering related to daily life events detected by context recognition systems, such as activity recognition and indoor positioning systems, e.g., answering questions like "Did my grandma eat dinner?" and "How many times did my grandpa go to the toilet?" These questions can effectively support human memory-aids, locate lost items, and monitor human activities. However, training a question-answering model requires large amounts of labeled training data (i.e., questions, answers, and the time-series of real-world event triplets) collected in a target environment. In this paper, we propose a novel simulation to real QA (Sim2RealQA) framework that completely trains a QA model with QA datasets produced in a life simulator and use it for solving real-word QA problems without answer labels. Our proposed QA model can learn a general reasoning process for QA that is independent of environments and deal with diverse types of questions specific to question answering in real-world environments, e.g., counting the number of occurrences of a real-world event and enumerating the names of those who are performing an activity together. Experiments show that using life simulations is a promising approach for solving real-world QA problems when no real-world answer labels are available.INDEX TERMS Lifelog, ubiquitous computing, daily-living activities, real-world question answering.

Synthesizing Event-Centric Knowledge Graphs of Daily Activities Using Virtual Space

“…Several studies also reported on the improvement of the accuracy of several real-world tasks using data in virtual space. Miyanishi et al [37] proposed an approach for improving the accuracy of real-world question answering tasks using the data obtained from a life simulation game. Hwang et al [38] developed a simulation platform that focused on the daily activities of older adults and used synthetic data to improve the action recognition accuracy.…”