Empirical validation of a usability inspection method for model-driven Web development

Abstract

Web applications should be usable in order to be accepted by users and to improve the success probability. Despite the fact that this requirement has promoted the emergence of several usability evaluation methods, there is a need for empirically validated methods that provide evidence about their effectiveness and that can be properly integrated into early stages of Web development processes. Model-driven Web development processes have grown in popularity over the last few years, and offer a suitable context in which to perform early usability evaluations due to their intrinsic traceability mechanisms. These issues have motivated us to propose a Web Usability Evaluation Process (WUEP) which can be integrated into model-driven Web development processes. This paper presents a family of experiments that we have carried out to empirically validate WUEP. The family of experiments was carried out by 64 participants, including PhD and Master's computer science students. The objective of the experiments was to evaluate the participants' effectiveness, efficiency, perceived ease of use and perceived satisfaction when using WUEP in comparison to an industrial widely-used inspection method: Heuristic Evaluation (HE). The statistical analysis and meta-analysis of the data obtained separately from each experiment indicated that WUEP is more effective and efficient than HE in the detection of usability problems. The evaluators were also more satisfied when applying WUEP, and found it easier to use than HE. Although further experiments must be carried out to strengthen these results, WUEP has proved to be a promising usability inspection method for Web applications which have been developed by using model-driven development processes.

Usability Inspection in Model-Driven Web Development: Empirical Validation in WebML

“…Although the evaluators found it easier to use than HE and they were also more satisfied when applying WUEP, these variables resulted not statistically significant in some cases. These results confirmed our previous findings [12] when an instantiation of WUEP into the OO-H method was compared against HE, strengthening the case for using WUEP rather than HE, at least in contexts with fairly inexperienced usability evaluators. Although the experimental results provided good results on the usefulness of WUEP as a usability inspection method for Web applications developed through MDWD processes, we are aware that more experimentation is needed to confirm these results, since they need to be interpreted with caution being them only valid within the context established in these experiments.…”

“…This motivated us to conduct a family of experiments to validate our usability inspection method when it was applied to the Object-Oriented Hypermedia (OO-H) method [12]. However, generalizations about the usefulness of WUEP require it to be instantiated and validated in other model-driven Web development methods.…”

“…However, they focus on traditional Web development processes, while only few studies address modeldriven Web development processes (e.g., [1][19] [27]). Among these studies, we presented in [12] an operationalization and validation of WUEP in a specific process based on the Object-Oriented Hypermedia (OO-H) method. In this study, WUEP was compared against Heuristic Evaluation (HE) [25].…”

A proposal for modelling usability in a holistic MDD method

“…There are also some works [12][22] related to measuring the system usability in MDD conceptual models. Fernandez [12] proposes a model to evaluate system usability from conceptual models.…”

“…Fernandez [12] proposes a model to evaluate system usability from conceptual models. According to Fernandez, the evaluation performed at the conceptual model level produces a platform-independent usability report, which provides feedback to the system analysis stage.…”