Examining Question-Answering Technology from the Task Technology Fit Perspective

Abstract

The World Wide Web has become a vital supplier of information for organizations in order to carry on such tasks as business intelligence, security monitoring, and risk assessments. By utilizing the task-technology fit (TTF) theory, we investigate the issue of when open-domain question-answering (QA) technology would potentially be superior to general-purpose Web search engines. Specifically, we argue theoretically and back up our arguments with a user study that the presence of fusion (information synthesis) is crucial to warrant the use of QA. At the same time, many information seeking tasks do not require fusion and, thus, are adequately served by traditional keyword search portals (Google, MSN, Yahoo, etc.). This explains why prior attempts to demonstrate the value of QA empirically were unsuccessful. We also discuss methodological challenges to any empirical investigation of QA and present several solutions to those challenges, validated with our user study. In order to carry our study, we created a novel prototype by following the Design Science guidelines. Our prototype is the first of its kind and is capable of answering list questions, such as What companies own low orbit satellites? or In which cities have illegal methylmethionine labs been found? This investigation is only a precursor to a full-scale empirical study, but it serves as a medium to overview the state of the art QA technologies and to introduce important theoretical and empirical concepts involved. Although we did not find empirical evidence that one technology is uniformly better than the other, we discovered that once the user accumulates experience using QA, he/she can make an intelligent decision whether to use it for a particular task, which leads to the user to be more productive on average with the same tasks compared to when there is no choice of technology.

What factors satisfy e-book store customers? Development of a model to evaluate e-book user behavior and satisfaction

“…A number of studies have investigated different topics and a wide range of tasks using the TTF model (Lu and Yang, 2014; Lin, 2012; Robles-Flores and Roussinov, 2012; Yuan et al , 2010; McGill and Klobas, 2009; Lee et al , 2007; Norzaidi et al , 2007; D’Ambra and Rice, 2001; Mathieson and Keil, 1998). One study, for example, examined the relationship between TTF and perceived ease of use with the system interface as the main factor (Mathieson and Keil, 1998).…”

“…One study, for example, examined the relationship between TTF and perceived ease of use with the system interface as the main factor (Mathieson and Keil, 1998). Others included evaluations of the following: user usage, user satisfaction, individual performance, and the impact after using the World Wide Web (D’Ambra and Rice, 2001); the impact of intranet usage on middle managers’ performance in the port industry (Norzaidi et al , 2007); TTF using mobile devices in the insurance industry (Lee et al , 2007); the ideal fit between mobile tasks and mobile work support (Yuan et al , 2010); and the advantages and shortcomings of open-domain question-answering technology and general-purpose web search engines (Robles-Flores and Roussinov, 2012). In addition, TTF has been applied in education research, such as in the evaluation of learning management systems (McGill and Klobas, 2009) and exploration of the intention to continue usage and the perceived fit of virtual learning systems (Lin, 2012).…”

Question answering systems for health professionals at the point of care—a systematic review

“…Several studies highlight obstacles that should be overcome and measures that should be taken before deploying biomedical QA systems. For example, one general-domain QA user study 115 found that users tended to prefer conventional search engines as they “felt less cognitive load” and “were more effective with it” than when they queried QA systems.…”

Strategies for Improving the Efficacy of Fusion Question Answering Systems

“…To the researcher's knowledge, this is the first study to compare the results of participants that used both QA and KW algorithms. The experiment is described in depth in Robles-Flores (Robles-Flores, 2009).…”