A study of student strategies for the corrective maintenance of concurrent software

Abstract

Graduates of computer science degree programs are increasingly being asked to maintain large, multi-threaded software systems; however, the maintenance of such systems is typically not wellcovered by software engineering texts or curricula. We conducted a think-aloud study with 15 students in a graduate-level computer science class to discover the strategies that students apply, and to what effect, in performing corrective maintenance on concurrent software. We collected think-aloud and action protocols, and annotated the protocols for a number of behavioral attributes and maintenance strategies. We divided the protocols into groups based on the success of the participant in both diagnosing and correcting the failure. We evaluated these groups for statistically significant differences in these attributes and strategies.In this paper, we report a number of interesting observations that came from this study. All participants performed diagnostic executions of the program to aid program comprehension; however, the participants that used this as their predominant strategy for diagnosing the fault were all unsuccessful. Among the participants that successfully diagnosed the fault and displayed high confidence in their diagnosis, we found two commonalities. They all recognized that the fault involved the violation of a concurrent-programming idiom. And, they all constructed detailed behavioral models (similar to UML sequence diagrams) of execution scenarios. We present detailed analyses to explain the attributes that correlated with success or lack of success. Based on these analyses, we make recommendations for improving software engineering curriculums by better training students how to apply these strategies effectively.

How Developers Use Data Race Detection Tools

“…For his thesis, Scott Fleming observed programmers debugging a multithreaded server application that had been seeded with a concurrency bug [10,11]. Interestingly, several participants were able to find and fix the flaw, but were not able to correctly describe the design defect that caused the problem.…”

Abstraction and mining of traces to explain concurrency bugs

“…Inspecting concurrent traces manually, however, is still tedious and timeconsuming. An empirical study of strategies commonly used for diagnosing and correcting faults in concurrent software shows that the primary concern of the programmer is to produce and analyze a failing trace by reasoning about potential thread interleavings based on some degree of program understanding [9]. In light of the complexity of this task, tool support is highly desirable.…”

Refining existing theories of program comprehension during maintenance for concurrent software

“…However, much less is known of issues specific to the maintenance of these systems. Previously, we performed a think-aloud study [16] of programmers engaged in the corrective maintenance of a concurrent program [7]. This study strove to understand what behaviors correlate with success on task.…”

“…In addition to the think-aloud data we collected and used for this prior study, the participants took a posttest to assess the extent to which they understood the program. However, at the time of publication [7], we had yet to analyze these data. This paper reports the results of the posttest analysis and attempts to reconcile these results with the predictions of two well-known studies [10,17].…”