A user model to directly compare two unmodified interfaces: a study of including errors and error corrections in a cognitive user model

Abstract

User models that can directly use and learn how to do tasks with unmodified interfaces would be helpful in system design to compare task knowledge and times between interfaces. Including user errors can be helpful because users will always make mistakes and generate errors. We compare three user models: an existing validated model that simulates users’ behavior in the Dismal spreadsheet in Emacs, a newly developed model that interacts with an Excel spreadsheet, and a new model that generates and fixes user errors. These models are implemented using a set of simulated eyes and hands extensions. All the models completed a 14-step task without modifying the system that participants used. These models predict that the task in Excel is approximately 20% faster than in Dismal, including suggesting why, where, and how much Excel is a better design. The Excel model predictions were compared to newly collected human data (N = 23). The model’s predictions of subtask times correlate well with the human data (r2 = .71). We also present a preliminary model of human error and correction based on user keypress errors, including 25 slips. The predictions to data comparison suggest that this interactive model that includes errors moves us closer to having a complete user model that can directly test interface design by predicting human behavior and performing the task on the same interface as users. The errors from the model’s hands also allow further exploration of error detection, error correction, and different knowledge types in user models.

Applications of artificial intelligence and cognitive science in design

“…This study has suggested several design recommendations for improving users' feeling of control in virtual object translation on 2D screens to better support creative idea generation. Tehranchi et al (2023) proposed a cognitive user model for testing interface design by predicting human behavior and performing relevant tasks. Computational models for stimulating human intelligence are often incomplete and lack interactions with the environment.…”

“…This thematic collection encompasses a wide range of studies exploring various aspects of applying AI techniques and cognitive science in design. Some studies have focused on how AI can be used to augment and support humans in design (Sun et al, 2023a;Wang et al, 2023), some explored new user cognitive models for design (Tehranchi et al, 2023), while others probed human design cognition using neurophysiological and biometric measures (Hu et al, 2023;Sun et al, 2023b;Yin et al, 2023;Zhang et al, 2023). Additional studies have exploited AI for analyzing design cognitive data (Chiu et al, 2023;Hu et al, 2023;Sun et al, 2023b;Zhang et al, 2023).…”

“…In this thematic collection, Wang et al (2023) assessed the creativity of AI-generated designs by using both human experts and computational techniques, Tehranchi et al (2023) showed the importance of human behavior predictions in computational models for design, and Sun et al (2023a) indicated the inclusion of human designers in ML lifecycles could better support the generation of design solutions. These three studies have shown the significance and potential necessity of integrating human expertise and intelligence into AI design process.…”