Evaluation criteria for information quality research

Abstract

Evaluation of research artefacts (such as models, frameworks and methodologies) is essential to determine their quality and demonstrate worth. However, in the IQ research domain there is no existing standard set of criteria available for researchers to use to evaluate their IQ artefacts. This paper therefore describes our experience of selecting and synthesizing a set of evaluation criteria used in three related research areas of Information Systems (IS), Software Products (SP) and Conceptual Models (CM), and analysing their relevance to different types of IQ research artefact. We selected and used a subset of these criteria in an actual evaluation of an IQ artefact to test whether they provide any benefit over a standard evaluation. The results show that at least a subset of the criteria from the other domains of IS, SP and CM are relevant for IQ artefact evaluations, and the resulting set of criteria, most importantly, enabled a more rigorous and systematic selection of what to evaluate.