A framework and methodology for studying the causes of software errors in programming systems

Abstract

An essential aspect of programmers' work is the correctness of their code. This makes current HCI techniques ill-suited to analyze and design the programming systems that programmers use everyday, since these techniques focus more on problems with learnability and efficiency of use, and less on error-proneness. We propose a framework and methodology that focuses specifically on errors by supporting the description and identification of the causes of software errors in terms of chains of cognitive breakdowns. The framework is based on both old and new studies of programming, as well as general research on the mechanisms of human error. Our experiences using the framework and methodology to study the Alice programming system have directly inspired the design of several new programming tools and interfaces. This includes the Whyline debugging interface, which we have shown to reduce debugging time by a factor of 8 and help programmers get 40% further through their tasks. We discuss the framework's and methodology's implications for programming system design, software engineering, and the psychology of programming. r

How Workstyle Transitions Influence the Tools' Perceptions of Use

“…ArgoUML or MS Visio, but they would have to be adapted to ensure they supported the same notations equally. We examined the usage of TaskSketch in a field study that used subjects enrolled in an undergraduate 3rd year Human-Computer Interaction course at the University of Madeira in a similar procedure as (Ko and Myers, 2005;Abrahão et al, 2005). In this section, we describe in detail the experiment's procedure, variables, results and discussion.…”

“…We devised an approach that aimed for an empirically sound framework that could be used to inform as well as to validate the design of new tools. This research objective is not new, but the question is very timely: the argument that the practitioner's behaviour must be studied in order to make better tools has been used recently by Seffah and Kline (2002); Ko and Myers (2005). There has also been a growing body of research aimed at building models for e.g.…”

Usability Experiments to Evaluate UML/SysML-Based Model Driven Software Engineering Notations for Logic Control in Manufacturing Automation

“…Real applications and some applications engineers are included in [85]. The classical debugging phase to find faults is not explicitly analyzed up to now even if Myers [86] provides an interesting approach to classify runtime faults and the underlying software errors. Debugging in E5 was limited to simulation and restricted due to given time.…”

How Developers Visualize Compiler Messages: A Foundational Approach to Notification Construction

“…But unlike C++, final variables in Java can be assigned at a point other than the declaration. Yoonki has experienced what we could call a knowledge breakdown [3]. In this case, Yoonki has a confirmation bias about how the system is supposed to work, and this false hypothesis has worked reasonably well for him until now.…”

“…Yet, for developers to comprehend the notification, they must mentally duplicate this process through selfexplanation [2] in essentially reverse order -starting with the error notification, the developer must identify what they think the problem might be from the IDE's presentation, mentally collect all of the program components related to this problem, and finally identify the area or areas of source code necessary to correct the particular defect. This self-explanation process, when left solely to the developer, can result in a significant loss of productivity because humans are imperfect and bounded in knowledge, attention, and expertise [3]. Much of this selfexplanation process may be completely unnecessary since the reasoning process that resulted in the error notification was already known to the compiler.…”