Professional status and expertise for UML class diagram comprehension: An empirical study

Abstract

Abstract-Professional experience is one of the most important criteria for almost any job offer in software engineering. Professional experience refers both to professional status (practitioner vs. student) and expertise (expert vs. novice). We perform an experiment with 21 subjects including both practitioners and students, and experts and novices. We seek to understand the relation between the speed and accuracy of the subjects and their status and expertise in performing maintenance tasks on UML class diagrams. We also study the impact of the formulation of the maintenance task. We use an eye-tracking system to gather the fixations of the subjects when performing the task. We measure the subjects' comprehension using their accuracy, the time spent, the search effort, the overall effort, and the question comprehension effort. We found that (1) practitioners are more accurate than students while students spend around 35 percent less time than practitioners, (2) experts are more accurate than novices while novices spending around 33 percent less time than experts, (3) expertise is the most important factor for accuracy and speed, (4) experienced students are more accurate and spend around 37 percent less time than experienced practitioners, and (5) when the description of the task is precise, the novice students can be accurate. We conclude that it is an illusion for project managers to focus on status only when recruiting a software engineer. Our result is the starting point to consider the differences between status and expertise when studying software engineers' productivity. Thus, it can help project managers to recruit productive engineers and motivated students to acquire the experience and ability in the projects.

A systematic literature review on the usage of eye-tracking in software engineering

“…Also, they tend to explore the diagrams from their centers whereas novices use either 840 top-down or left-to-right strategies. [Soh et al, 2012] discusses the differences between participants' status (practitioners or students) and expertise (experts or novices) when [ Sharafi et al, 2013] reports that participants spend more time and effort while working with a graphical representation than a textual representa-895 tion of some requirements, although no significant difference is reported for accuracy and participants state that they prefer the graphical representation. Moreover, it reports that the spatial structure of the graphical representation facilitates the compre-900 hension tasks and leads participants to follow two different navigation strategies (top-down and bottom up) to perform the comprehension task.…”

“…To understand the impact of multiple representations on debugging [Jermann and Nüssli, 2012] To study the impact of sharing selection among collaborators in a pairprogramming task [Sharafi et al, 2012] To investigate the impact of gender on developers' effectiveness for source code reading and identifier recalling [Sharif et al, 2012] To study the impact of scan Time in detecting source code defects [Soh et al, 2012] To analyze the impact of expertise on developers' speed and accuracy [Binkley et al, 2013] To analyze the impact of identifier style on program comprehension [Cagiltay et al, 2013] To propose measures to enhance understanding of ERD diagrams [Duru et al, 2013] To study the impact of software visualization in program comprehension [Petrusel and Mendling, 2012] To understand the impact of multiple representations in debugging [Sharafi et al, 2013] To study the relations between the type of requirement representations (graphical vs. textual) and developers efficiency [Sharif et al, 2013] To analyze the impact of SeeIT 3D on software engineering tasks [Sharma et al, 2013] To study the interaction of developers in a pair programming task [58] To recover traceability links from eye-movements data [Busjahn et al, 2014] To present a method to recover traceability links from eye-gaze data [Fritz et al, 2014] To use psycho-physiological measures to calculate task difficulty [Rodeghero et al, 2014]…”

On the impact of size to the understanding of UML diagrams

“…More recently, there has been the study of Soh et al [46], who use eye tracking in a large empirical study (n=21), though they restrict themselves to class diagrams, and are interested in modeler performance only with a view to its predictive value for success in a future professional career. Sharif et al [40] study 3 different layout schemes of UML class diagrams using both questionnaires and eye tracking on maintenance tasks.…”

Assessing the use of slicing-based visualizing techniques on the understanding of large metamodels

“…Empirical studies have been conducted on the understanding of UML class diagrams when doing maintenance activities on class diagrams [60], or depending on the layout used [61,62]. In particular, Purchase et al highlight that when visualizing domain-specific graphs (e.g.…”