CogTool-Explorer

Abstract

CogTool-Explorer 1.2 (CTE1.2) predicts novice exploration behavior and how it varies with different user-interface (UI) layouts. CTE1.2 improves upon previous models of information foraging by adding a model of hierarchical visual search to guide foraging behavior. Built within CogTool so it is easy to represent UI layouts, run the model, and present results, CTE1.2's vision is to assess many design ideas at the storyboard stage before implementation and without the cost of running human participants. This paper evaluates CTE1.2 predictions against observed human behavior on 108 tasks (36 tasks × 3 distinct website layouts). CTE1.2's predictions accounted for 63-82% of the variance in the percentage of participants succeeding on each task, the number of clicks to success, and the percentage of participants succeeding without error. We demonstrate how these predictions can be used to identify areas of the UI in need of redesign.CTE1.2 is a research extension of CogTool that connects a computational model of eye-movements, visual perception, cognition and motor actions (i.e., a simulated user) with a UI storyboard. The original CogTool [13] had only a simulation of a skilled user (i.e., a Keystroke-Level Model [6]). The first version of CogTool-Explorer [24] added a model of novice behavior that considered 2-D layout to CogTool so a designer can make predictions of novice exploration on the same tasks and UIs that he or she

An open source approach to information scent

“…To begin to answer this question, we compared a previously published, but proprietary infoscent generator (ProprietaryInfoscent, AutoCWW's Latent Semantic Analysis implementation with the first-yearcollege-level TASA corpus [1,7,8]) to an open source Work-in-Progress: Evaluation and Design Methods CHI 2013: Changing Perspectives, Paris, France LSA algorithm, Gensim [6], creating an LSA semantic space from an easily available open source corpus, Simple English Wikipedia (OpenSourceInfoscent). We chose Gensim as the LSA algorithm because it is programmed in Python, a relatively easy language for non-computer scientists to use and thus a good code base for HCI researchers.…”

“…Both ProprietaryInfoscent and OpenSourceInfoscent provide estimates of infoscent approximated by the cosine between "documents" in the LSA vector space. These estimates of infoscent can then be used by tools like AutoCWW [1] and CogTool-Explorer v1.2 (CTE1.2) [7] to predict what percentage of people will succeed on a previously-published set of information foraging tasks. We have begun our work using CTE1.2 because it has been shown to predict the effects of page layout in addition to those of the infoscent of links and group headings [7].…”

“…We used the 36 tasks from [7] where a participant was asked to find an encyclopedia article about an item (presented at the top of a webpage shown in Figure 1), in a page of 93 links nested under 9 group headings. Clicking on a link brought up a page with an alphabetic list of articles where the participant could find the target item, or hit the browser's back button to go back to the top-level page and try another link.…”

“…We use the human data from [7], where between 36 and 60 participants were asked to click links until they either successfully found the correct webpage or were shown a webpage announcing that time had run out (after 130 sec). The tasks alternated between hard and easy tasks, counterbalancing to prevent order effects.…”

“…For example, the Comprehensionbased Linked-model of Deliberate Search (CoLiDeS) theory [3] led to Automated Cognitive Walkthrough for the Web (AutoCWW) [1] and Information Foraging Theory [5] led to SNIF-ACT [2] and CogTool-Explorer [7]. Useful as these theories and tools may be, the field of HCI would like to have generalizable methods and tools that can be used to evaluate a broad range of interfaces, from desktops to smartphones, from invehicle devices in the automotive industry to flight automation systems in the cockpit.…”