Analyzing a socio-technical visualization tool using usability inspection methods

Abstract

Ariadne is a novel visualization tool that allows end users to explore the socio-technical relationships in software development projects. Essentially the visualization is a variant of a social network graph. It is based on the observation that dependencies between software components create dependencies between the developers implementing those components. This relationship emerged in our own and other researchers' field studies of software projects. Large software development projects require management of dependencies by managers and developers to ensure the smooth coordination of work. We sought to evaluate our visualization to assess its utility. Although we had some informal trials with potential end users, we sought a deeper analysis before further refinement of the tool and evaluation on a larger scale. Usability inspection methods provided one potential avenue. Moreover, such inspection methods yield a kind of rationale not directly derived from human subjects evaluations. We report on the application of these inspection methods and discuss the implications of their results in the context of usability evaluations for visual interfaces.

Continuous Coordination Tools and their Evaluation

“…Usability is especially important in the context of the work discussed here because of the upfront costs associated with human subjects testing. Future WAV evaluations involves the application of usability inspection methods such as those applied to Ariadne [51]. These evaluations can reveal patterns of interest and compare activity between both developers and artifacts.…”

“…It draws connections from a dependent author to the code unit they are dependent upon and back to the author responsible for that code unit and repeats this for each code unit in the project. Further details on its visualization and its advantages have been reported elsewhere [51].…”

“…This unfamiliarity helped us to identify problematic design assumptions about new users' expectations and assumptions about interacting with and drawing conclusions from the visualization. Further information of our evaluations is detailed elsewhere [51].…”

Developing guidelines for assessing visual analytics environments

“…A number of visualization researchers have looked at heuristic evaluations for visualizations. [10][11][12][13][14][15][16] While the consensus is that heuristics are useful, there is considerable work that must be done before an agreed upon set of heuristics exists. First of all, we need to start with lower level guidelines.…”