Rapid Evaluation Methods

Hartson, Rex; Pyla, Partha S.

doi:10.1016/b978-0-12-385241-0.00013-0

Cited by 4 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thirdly, the evaluation results may lack actionable suggestions for resolving identified usability issues, potentially necessitating additional data collection [34]. Finally, the original scoring system introduced by Nielsen and Molich [14] has often perplexed evaluators, with its differentiation among severity, frequency, and criticality attributes, thus heuristic methods without a rigid framework poorly support problem discovery [35]. This confusion has led many experts to predominantly focus on one attribute, typically criticality, highlighting the need for subsequent proposals to refine this aspect [36,37].…”

Section: Evaluators Biasesmentioning

confidence: 99%

Towards a Refined Heuristic Evaluation: Incorporating Hierarchical Analysis for Weighted Usability Assessment

Talero-Sarmiento,

Gonzalez-Capdevilla,

Granollers

et al. 2024

Preprint

View full text Add to dashboard Cite

This study delves into the realm of heuristic evaluation within usability testing, focusing on refining and applying a tailored algorithm that simplifies the implementation of the Analytic Hierarchy Process for heuristic prioritization. Addressing the prevalent challenge of disparate evaluation methodologies and the absence of a standardized process in usability testing, our research introduces a novel method that leverages the Analytic Hierarchy Process alongside a bespoke algorithm. This algorithm utilizes transitive properties for pairwise comparisons, significantly reducing the evaluative workload. This innovative approach facilitates the estimation of heuristic relevance irrespective of the number of items per heuristic or the item scale and streamlines the overall evaluation process. In addition to rigorous simulation testing of the tailored algorithm, we applied our method in a practical scenario, engaging seven usability experts in evaluating a web interface. This empirical application underscored our method’s capacity to diminish the requisite comparison count and spotlight areas of improvement for critically weighted yet underperforming heuristics. The outcomes of this study underscore the efficacy of our approach in enhancing the efficiency and organization of heuristic evaluations, paving the way for more structured and effective usability testing methodologies in future research endeavors.

show abstract

Section: Evaluators Biasesmentioning

confidence: 99%

Towards a Refined Heuristic Evaluation: Incorporating Hierarchical Analysis for Weighted Usability Assessment

Talero-Sarmiento,

Gonzalez-Capdevilla,

Granollers

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Once the second prototype was mature enough, some sessions of evaluation, based on the quasi-empirical evaluation approach [5] were organized with three users who were not involved in the earlier development. These three users were all social scientists, with extensive expertise of manual and semi-automatic text analysis techniques.…”

Section: Preliminary Feedbackmentioning

confidence: 99%

Shock wave

Cauz

Albert

Wallemacq

et al. 2021

Proceedings of the 21st ACM Symposium on Document Engineering

View full text Add to dashboard Cite

Figure 1: A graph layout in the EVOQ tool with on the left a random initial placement of nodes and on the right a initial placement done by the Shock Wave. As can be seen, our algorithm helps to distinguish the different semantic fields extracted by the user (i.e. groups of nodes linked by association relations association (green links)). And it highlights opposition relations (red links) which are a key part for the interpretation of these semantic fields.

show abstract

“…Thirdly, the evaluation results may lack actionable suggestions for resolving identified usability issues, potentially necessitating additional data collection [39]. Finally, the original scoring system introduced by Nielsen and Molich [13] has often perplexed evaluators due to its differentiation among severity, frequency, and criticality attributes; thus, heuristic methods without a rigid framework poorly support problem discovery [40]. This confusion has led many experts to predominantly focus on one attribute, typically criticality, highlighting the need for subsequent proposals to refine this aspect [41,42].…”

mentioning

confidence: 99%

Towards a Refined Heuristic Evaluation: Incorporating Hierarchical Analysis for Weighted Usability Assessment

Talero-Sarmiento,

Gonzalez-Capdevila,

Granollers

et al. 2024

BDCC

View full text Add to dashboard Cite

This study explores the implementation of the analytic hierarchy process in usability evaluations, specifically focusing on user interface assessment during software development phases. Addressing the challenge of diverse and unstandardized evaluation methodologies, our research develops and applies a tailored algorithm that simplifies heuristic prioritization. This novel method combines the analytic hierarchy process framework with a bespoke algorithm that leverages transitive properties for efficient pairwise comparisons, significantly reducing the evaluative workload. The algorithm is designed to facilitate the estimation of heuristic relevance regardless of the number of items per heuristic or the item scale, thereby streamlining the evaluation process. Rigorous simulation testing of this tailored algorithm is complemented by its empirical application, where seven usability experts evaluate a web interface. This practical implementation demonstrates our method’s ability to decrease the necessary comparisons and simplify the complexity and workload associated with the traditional prioritization process. Additionally, it improves the accuracy and relevance of the user interface usability heuristic testing results. By prioritizing heuristics based on their importance as determined by the Usability Testing Leader—rather than merely depending on the number of items, scale, or heuristics—our approach ensures that evaluations focus on the most critical usability aspects from the start. The findings from this study highlight the importance of expert-driven evaluations for gaining a thorough understanding of heuristic UI assessment, offering a wider perspective than user-perception-based methods like the questionnaire approach. Our research contributes to advancing UI evaluation methodologies, offering an organized and effective framework for future usability testing endeavors.

show abstract

Rapid Evaluation Methods

Cited by 4 publications

References 0 publications

Towards a Refined Heuristic Evaluation: Incorporating Hierarchical Analysis for Weighted Usability Assessment

Towards a Refined Heuristic Evaluation: Incorporating Hierarchical Analysis for Weighted Usability Assessment

Shock wave

Towards a Refined Heuristic Evaluation: Incorporating Hierarchical Analysis for Weighted Usability Assessment

Contact Info

Product

Resources

About