Information Artifact Evaluation with TEDSrate

Abstract

The evaluation of systems or artifacts as "outcomes" of software engineering (SE) projects has been a focus of study in SE-related research for quite some time. In recent years, evaluating artifacts, for example, mobile applications or websites has become more important, since such artifacts play increasingly critical roles in generating revenues for businesses, and the degree of artifact effectiveness is seen as a competitive factor. With the TEDS framework/procedure a novel and comprehensive approach to systematic artifact evaluation and comparison had been presented a few years ago, whose effectiveness and analytical power in comprehensive and highly detailed artifact evaluations and comparisons was empirically shown; however, despite its demonstrated capability TEDS still proved to be time and resource consuming like other evaluation approaches before. In order to overcome these constraints and provide evaluative feedback more quickly to developers and service providers, TEDSrate, a Web-based evaluation tool employing the TEDS framework/procedure, was developed. The tool was tested with two real-world organizations, the City of Seattle Emergency Operations Center (EOC) and the Seattle Sounders Football Club. The tests suggest that the highly configurable TEDSrate tool can fully implement and administer the TEDS framework/procedure and, at the same time, provide instantaneous, cost-effective, comprehensive, and highly detailed artifact evaluations to both developers and service providers.