On Pairwise Comparisons Based Internal and External Measures for Software Evaluation

Abstract

We show how pairwise comparisons paradigm can be used to improve many current methods of software evaluation. Most of software metrics lack formal analysis of weights consistency, while inconsistency analysis is the crucial part of all decision making techniques based on the pairwise comparisons. We show how distance-based inconsistency, a measure of inconsistency often used when pairwise comparisons approach is applied, can improve accuracy of weights assignment for attributes used for software evaluation. If the phenomena are not well identified or the values and their ranges in the models dynamics are not known the external measures (weights) are introduced. While weights are external measures of attributes importance, pairwise comparisons explore their mutual internal structure.