Modeling programmer navigation: A head-to-head empirical evaluation of predictive models

Abstract

Software developers frequently need to perform code maintenance tasks, but doing so requires time-consuming navigation through code. A variety of tools are aimed at easing this navigation by using models to identify places in the code that a developer might want to visit, and then providing shortcuts so that the developer can quickly navigate to those locations. To date, however, only a few of these models have been compared head-to-head to assess their predictive accuracy. In particular, we do not know which models are most accurate overall, which are accurate only in certain circumstances, and whether combining models could enhance accuracy. Therefore, we have conducted an empirical study to evaluate the accuracy of a broad range of models for predicting many different kinds of code navigations in sample maintenance tasks. Overall, we found that models tended to perform best if they took into account how recently a developer has viewed pieces of the code, and if models took into account the spatial proximity of methods within the code. We also found that the accuracy of single-factor models can be improved by combining factors, using a spreadingactivation based approach, to produce multi-factor models. Based on these results, we offer concrete guidance about how these models could be used to provide enhanced software development tools that ease the difficulty of navigating through code.

Tracing software developers' eyes and interactions for change tasks

“…Our results on switches to methods in close proximity further support the findings of a recent head-to-head study that compared different models of a programmer's navigation [29] and that suggested to use models to approximate a developer's navigation based on the spatial proximity of methods within the source code.…”

“…Based on such automatically captured interaction histories-logs of the code elements a developer interacted with along with a timestamp-researchers have, for instance, investigated how developers work in an IDE [27], how they navigate through code [28,29,47], or how developers' micro interaction patterns might be used for defect prediction [24]. Even the Eclipse team themselves undertook a major data collection project called the Usage Data Collector that, at its peak, collected data from thousands of developers using Eclipse.…”

“…To evaluate whether gaze and interaction contexts capture different aspects of a developer's navigation for change task investigations, we defined ranking models based on the data available in the different contexts and compared the top ranked methods. There are a variety of models that can be used to select the most important elements within a navigation sequence [29]. For our analysis, we used single-factor models to select the most important elements in each kind of context that were also suggested in previous studies [28,29].…”

“…There are a variety of models that can be used to select the most important elements within a navigation sequence [29]. For our analysis, we used single-factor models to select the most important elements in each kind of context that were also suggested in previous studies [28,29]. To rank the methods of a gaze context we used a time-based model.…”

“…Since the inception of software development, researchers have been studying how developers read and navigate code, and what kind of knowledge they acquire (e.g., [43,5,22]). The more we know about a developer's work, the better we are able to support her, for instance, by reducing information overload [21], improving defect prediction [24], or providing automatic navigation recommendations [13,29].…”