An empirical study on the efficiency of graphical vs. textual representations in requirements comprehension

Abstract

Abstract-Graphical representations are used to visualise, specify, and document software artifacts in all stages of software development process. In contrast with text, graphical representations are presented in two-dimensional form, which seems easy to process. However, few empirical studies investigated the efficiency of graphical representations vs. textual ones in modelling and presenting software requirements. Therefore, in this paper, we report the results of an eye-tracking experiment involving 28 participants to study the impact of structured textual vs. graphical representations on subjects' efficiency while performing requirement comprehension tasks. We measure subjects' efficiency in terms of the percentage of correct answers (accuracy) and of the time and effort spend to perform the tasks.We observe no statistically-significant difference in term of accuracy. However, our subjects spent more time and effort while working with the graphical representation although this extra time and effort does not affect accuracy. Our findings challenge the general assumption that graphical representations are more efficient than the textual ones at least in the case of developers not familiar with the graphical representation. Indeed, our results emphasise that training can significantly improve the efficiency of our subjects working with graphical representations. Moreover, by comparing the visual paths of our subjects, we observe that the spatial structure of the graphical representation leads our subjects to follow two different strategies (top-down vs. bottomup) and subsequently this hierarchical structure helps developers to ease the difficulty of model comprehension tasks.

Graphical vs. Tabular Notations for Risk Models: On the Role of Textual Labels and Complexity

“…In the literature we found three main streams of works that compares textual and visual notations: a) studies that proposed cognitive theories to explain the differences between the notations or to explain their relative strengths (Vessey 1991), b) studies that compared different notations from a conceptual point of view (Kaczmarek et al 2015;Saleh and El-Attar 2015), and c) studies that empirically compare graphical and textual representations, e.g., for safety and system requirements (Sharafi et al 2013;Stålhane and Sindre 2008;Stålhane et al 2010;Stålhane and Sindre 2014;de la Vara et al 2016), software architectures (Heijstek et al 2011), and business processes (Ottensooser et al 2012). To the best of our knowledge, there are few similar studies that empirically investigated modeling notations for security risk (Hogganvik and Stølen 2005;Grøndahl et al 2011) or compared graphical and tabular security methods in full scale application experiments (Massacci and Paci 2012;Labunets et al 2013Labunets et al , 2014.…”

“…The results did not reveal any effect of using UML analysis models on understanding source code or ability to modify it. Similar to our paper, Sharafi et al (2013) investigated three requirements modeling notations w.r.t. requirements comprehension.…”

Model Comprehension for Security Risk Assessment: An Empirical Comparison of Tabular vs. Graphical Representations

“…Several empirical studies have compared graphical and textual representations for requirements (Sharafi et al 2013;Stålhane and Sindre 2008;Stålhane et al 2010;Stålhane and Sindre 2014), software architectures (Heijstek et al 2011), and business processes (Ottensooser et al 2012). Studies that focus on comparing textual and visual notations for security risk models are less frequent (Hogganvik and Stolen 2005;Grondahl et al 2011) or compared the effectiveness of tabular or graphical methodologies as whole (Massacci and Paci 2012;Labunets et al 2013Labunets et al , 2014b as opposed to the specific aspect of comprehensibility.…”

“…The authors explained the results by the fact that the provided UML models did not contain any details on the systems implementation, and therefore, not very helpful for understanding and modifying source code. Sharafi et al (2013) assessed the effect of using graphical vs. textual representations on participants' efficiency in performing requirements comprehension tasks. They found no difference in accuracy of the answers given by participants who used the textual and the graphical notations but it took them considerably more time to perform the task with a graphical notation than with textual one.…”

“…However, the design of the questionnaire may be strongly favoring one treatment over the other. Inspired by similar studies (Heijstek et al 2011;Sharafi et al 2013), we used the names of element types in the question statements.…”