A Paradox in Bland-Altman Analysis and a Binary Approach

Abstract

A reliable method of measurement is important in various scientific areas. When a new method of measurement is developed, it should be tested against a standard method that is currently in use. Bland and Altman proposed limits of agreement (LOA) to compare two methods of measurement under the normality assumption. Recently, a sample size formula has been proposed for hypothesis testing to compare two methods of measurement. In the hypothesis testing, the null hypothesis states that the two methods do not satisfy a pre-specified acceptable degree of agreement. Carefully considering the interpretation of the LOA, we argue that there are cases of an acceptable degree of agreement inside the null parameter space. We refer to this subset as the paradoxical parameter space in this article. To address this paradox, we apply a Bernoulli approach to modify the null parameter space and to relax the normality assumption on the data. Using simulations, we demonstrate that the change in statistical power is not negligible when the true parameter values are inside or near the paradoxical parameter space. In addition, we demonstrate an application of the sequential probability ratio test to allow researchers to draw a conclusion with a smaller sample size and to reduce the study time. MethodsThis section is structured as follows. The formulation of the hypothesis testing by Lu et al. (2016) is reviewed in Section 2.1. The hypothesis testing is formulated under the normality assumption, and it is referred to as the normal approach throughout the paper. In Section 2.2, a paradoxical case of the normal approach is introduced, and an alternative approach is considered in Section 2.3. In the new approach, a Bernoulli distribution (i.e., binary outcome) is used to address the paradox and to relax the normality assumption. We refer to this alternative approach as the Bernoulli approach throughout the paper. In Section 2.4, the required sample size is compared between the normal approach and the Bernoulli approach. One caveat of the Bernoulli approach is the minimum sample size requirement, and it is briefly discussed in Section 2.5. In Section 2.6, the application of sequential analysis (Wald, 1945;Wald, 1947) is discussed to overcome the caveat. Normal Approach (Lu et al., 2016)Let D i be the difference between two measurements for i = 1, 2, . . . , n, where n is a fixed sample size. Assume D 1 , . . . , D n are independent random variables, and assume D i ∼ N(µ, σ 2 ). Let A = µ − z 1−γ/2 σ be the true lower limit of an acceptable degree of agreement and B = µ + z 1−γ/2 σ be the true upper limit, where z 1−γ/2 is the 100(1 − γ/2)-th percentile of the standard normal distribution (e.g., z 0.975 = 1.96). The probabilistic interpretation of (Point estimators for A and B are =D − z 1−γ/2 S D andB =D + z 1−γ/2 S D , respectively, and their standard errors can be estimated byGiven a confidence level 1 − α, CIs for A and B areGiven a pre-defined acceptable limit, δ > 0, according to Lu et al. (2016), the null hypothesis is formulated as H...

A Tutorial of Bland Altman Analysis in A Bayesian Framework

“…In this case, θ 2 ¼ μ þ 1:96σ ¼ 0:1088 exceeds δ ¼ 0:1 (i.e., H 0 is true), but PðÀ δ < D i < δÞ ¼ 0:9522 exceeds 0.95 (which may be an acceptable probability of agreement). The two statements "H 1 : θ 1 � À δ and θ 2 � δ" and "PðÀ δ � D i � δÞ � 0:95" are not equivalent (Kim & Wand, 2020). The former statement is regarding the two parameters θ 1 ¼ μ À 1:96σ and θ 2 ¼ μ þ 1:96σ, and the latter statement is regarding the random variable D i .…”

“…Note that hypothesis testing and confidence intervals are used to make statements about unknown parameters (not future outcomes). Kim and Wand (2020) discussed a strange case in the hypothesis testing discussed in Section 3.2. For instance, let δ ¼ 0:1, and assume the true parameter values are μ ¼ 0:05 and σ ¼ 0:03.…”