On Jones et al.’s Method for Assessing Limits of Agreement with the Mean for Multiple Observers

Abstract

Background To assess the agreement of continuous measurements between a number of observers, Jones et al. introduced limits of agreement with the mean (LOAM) for multiple observers, representing how much an individual observer can deviate from the mean measurement of all observers. Besides the graphical visualisation of LOAM, suggested by Jones et al., it is desirable to supply LOAM with confidence intervals and to extend the method to the case of multiple measurements per observer.Methods We reformulate LOAM under the assumption the measurements follow an additive two-way random effects model. Assuming this model, we provide estimates and confidence intervals for the proposed LOAM. Further, this approach is easily extended to the case of multiple measurements per observer.Results The proposed method is applied on two data sets to illustrate its use. Specifically, we consider agreement between measurements regarding tumour size and aortic diameter. For the latter study, three measurement methods are considered. Conclusions The proposed LOAM and the associated confidence intervals are useful for assessing agreement between continuous measurements.

Clinical evaluation of AI-assisted muscle ultrasound for monitoring muscle wasting in ICU patients

“…There was no measure-remeasure variation for the automated AI software because the model always outputs the same measurement and hence the same RFCSA result. A modified Bland-Altman analysis for multiple observers in a single plot proposed by [26] was used to assess the agreement between RFCSA measurements. The examination duration was compared between the two methodologies of measurement using an unpaired Student's t-test.…”

Development and Application of a Web-Based Platform for Assessment of Observer Performance in Medical Imaging

“…In III and IV, for assessing agreement, we used reformulated LOAMs calculated based on first and second session measurements and equipped with an asymmetric and approximate CI. Based on a fixed number of subjects/patients and initial estimates of the intra-and inter-observer variation, Christensen et al provided a formula for determining how many observers are needed to obtain an expected width of the 95% CI for the 95% LOAM 68 . This formula was used in III.…”