Cohen’s Linearly Weighted Kappa is a Weighted Average of 2×2 Kappas

Abstract

An agreement table with n ∈ N ≥3 ordered categories can be collapsed into n − 1 distinct 2 × 2 tables by combining adjacent categories. Vanbelle and Albert (Stat. Methodol. 6:157-163, 2009c) showed that the components of Cohen's weighted kappa with linear weights can be obtained from these n − 1 collapsed 2 × 2 tables. In this paper we consider several consequences of this result. One is that the weighted kappa with linear weights can be interpreted as a weighted arithmetic mean of the kappas corresponding to the 2 × 2 tables, where the weights are the denominators of the 2 × 2 kappas. In addition, it is shown that similar results and interpretations hold for linearly weighted kappas for multiple raters.

Cohen’s linearly weighted kappa is a weighted average

“…In this paper we presented a strong basis for the use of weighted kappa with linear weights. The results presented here generalize the results derived in Vanbelle and Albert (2009c) and Warrens (2011b). An agreement table with n ≥ 3 ordered categories can for fixed m ∈ {2, 3, .…”

“…We show that the results presented in Vanbelle and Albert (2009c) and Warrens (2011b) describe a special case of a more general property of weighted kappa. An n × n agreement table F = { f i j } with n ≥ 3 ordered categories can for fixed m ∈ {2, 3, .…”

“…The theorems presented in this paper can also be formulated for the linearly weighted kappas for three or more raters presented in Mielke et al (2007Mielke et al ( , 2008 and Warrens (2011b). For example, for three raters the linear weight of the weighted kappa in Mielke et al (2007Mielke et al ( , 2008) is given by If we replace |i − j| = max(i, j) − min(i, j) by max(i, j, k) − min (i, j, k) in the proof of Theorem 1, then a result analogous to Theorem 1 for the linearly weighted kappa in Mielke et al (2007Mielke et al ( , 2008 follows almost immediately from using the same arguments.…”

“…Vanbelle and Albert (2009c) and Warrens (2011b) considered the case where the agreement table is collapsed into subtables of size 2 × 2. In Theorem 2 we consider, for a fixed value of m ∈ {2, .…”

“…Vanbelle and Albert (2009c) showed that the components (observed and expected agreement) of weighted kappa with linear weights can be obtained from the 2 × 2 subtables. A consequence is that the weighted kappa with linear weights can be interpreted as a weighted average of the 2 × 2 kappas, where the weights are denominators of the 2 × 2 kappas (Warrens 2011b).…”