An Exploratory Study of Debugging Episodes

Abstract

Many studies have long investigated how developers debug, shaping our understanding of debugging and helping motivate the creation of more effective tools. However, less is known about the typical progression of debugging in real world settings. In this study, we focus on characterizing debugging episodes from the moment at which developers first encounter a defect to the moment at which it is resolved. We investigate the typical duration and frequency of debugging episodes and the typical activities which occur. We observed developers by watching professional developers at work in live-streamed programming sessions. Using this data source, we curated 15 sessions in which 11 professional developers worked for 30 hours. We then systematically coded the debugging episodes and activities that occurred within these videos, yielding a dataset of 2137 debugging activities and 1407 programming activities. We found that debugging was frequent, even in programming work, occurring once every eight minutes. Debugging episodes vary greatly in time, with most being less than a few minutes and a few as more than 100 minutes. However, most debugging time is spent in long debugging episodes. We found no single activity that dominated debugging time, and long debugging episodes often involved many diverse activities. Finally, we found that,in terms of the activities developers did, programming and debugging were remarkably similar, particularly in the frequency of editing and browsing code.

Edit-Run Behavior in Programming and Debugging

“…The platform supports analysis of developer activity annotations alongside the recorded videos. The dataset and platform are described in detail in Alaboudi et al [20]. We briefly review the dataset here.…”

“…One potential threat to external validity is the representativeness of the developer activities that we analyzed. Our analysis of edit-run cycles drew from an existing dataset of professional developers at work [20]. All the videos in the dataset show professional developers with at least seven years of experience working on active open source projects.…”

“…To investigate these questions, we conducted the first empirical study of edit-run behavior. We used publicly available data to analyze 28 hours of work on open source projects by 11 professional developers [20]. Using this dataset, we constructed edit-run cycles that occurred in debugging and programming.…”

Error Identification Strategies for Python Jupyter Notebooks

“…The first phase involves identifying what and where the error is, and the second phase involves determining how best to fix the error [28]. We have many insights about the process of debugging software systems and programs in general [2,12,25], but we lack similar insights for computational notebooks. To the best of our knowledge, there is only one study looking at how computational notebooks are debugged, from Yang et al [27], and it focused on a tool for improving code understanding with program synthesis but did not synthesize strategies used in debugging.…”

“…Overall, they observed that developers found it challenging to formulate a reasonable hypothesis about a potential error. In the second paper, "An Exploratory Study of Debugging Episodes", Alaboudi and Latoza observed 15 live-streamed programming sessions (in C, C#, JavaScript) [2]. They found that developers spent 48% of their programming sessions debugging.…”