Variation of Preference Inconsistency When Applying Ratio and Interval Scale Pairwise Comparisons

Abstract

Several studies on numerical rating in discrete choice problems address the tendency of inconsistencies in decision makers' measured preferences. This is partly due to true inconsistencies in preferences or the decision makers' uncertainty on what he or she really wants. This uncertainty may be reflected in the elicited preferences in different ways depending on the questions asked and methods used in deriving the preferences for alternatives. Some part of the inconsistency is due to only having a discrete set of possible judgments. This study examined the variation of preference inconsistency when applying different pairwise preference elicitation techniques in a five-item discrete choice problem. The study data comprised preferences of five career alternatives elicited applying interval scale and numerically and verbally anchored ratio scale pairwise comparisons. Statistical regression technique was used to analyse the differences of inconsistencies between the tested methods. The resulting relative residual variances showed that the interval ratio scale comparison technique provided the greatest variation of inconsistencies between respondents, thus being the most sensitive to inconsistency in preferences. The numeric ratio scale comparison gave the most uniform preferences between the respondents. The verbal ratio scale comparison performed between the latter two when relative residual variances were considered. However, the verbal ratio scale comparison had weaker ability to differentiate the alternatives. The results indicated that the decision recommendation may not be sensitive to the selection between these preference elicitation methods in this kind of five-item discrete choice problem. The numeric ratio scale comparison technique seemed to be the most suitable method to reveal the decision makers' true preferences. However, to confirm this result, more studying will be needed, with an attention paid to users' comprehension and learning in the course of the experiment.

Reconstructing forest canopy from the 3D triangulations of airborne laser scanning point data for the visualization and planning of forested landscapes

“…According to our tests, even a small inconsistency in just one of the comparisons could increase the CR above the threshold of 0.1. Similar problems are reported in ratio-scale pairwise comparisons based on other types of surveys (Sironen et al 2013), and other measures have been presented for evaluating the inconsistencies (Peláez and Lamata 2003;Ramík and Korviny 2010) and uncertainties related to the process overall (see Alho and Kangas 1997), differing from those originally proposed by Saaty (1980). According to the earlier studies on the preferences on forest use, the majority of citizens, regardless of forest ownership, do not approve of clearcutting (Valkeapää and Karppinen 2013).…”

How to support the application of multiple criteria decision analysis? Let us start with a comprehensive taxonomy

“…Management of inconsistent preferences has received limited attention in the reviewed studies. It is though well known that the same DM might be inconsistent in his/her judgments, or the group providing the preferences may not unanimously agree with the same preferences [ 48 , 155 ]. As a consequence, it is desirable to have more MCDA methods that can deal with these types of decision context due to their inbuilt features.…”

Evaluating the role of spatial landscape literacy in public participation processes and opinions on environmental issues and ecosystem services

“…In terms of spatial cognition other measures of spatial knowledge, learning, and familiarity could have been included in our survey (Golledge 1993;Tu Huynh and Doherty 2007). Likewise, the use of Likert-scale levels, specific preference elicitation methods, and the time of survey implementation (December-January is a holiday season in Colombia) can have an influence on response rates and the statistical results (Sironen et al 2014). Finally, such response rates can be an indication of not only survey exhaustion or protest responses, but also respondent's lack of spatial knowledge and abilities to read a satellite image and maps.…”