Assessing Reliability and Precision of Measurement: An Introduction to Intraclass Correlation and Standard Error of Measurement

Abstract

The reliability and precision of measurement in sports medicine are of concern in both research and clinical practice. The validity of conclusions drawn from a research project and the rationale for decisions made about the care of an injured athlete are directly related to the precision of measurement. Through analysis of variance, estimates of reliability and precision of measurement can be quantified. The purpose of this manuscript is to introduce the concepts of intraclass correlation as an estimate of reliability and standard error of measurement as an estimate of precision. The need for a standardized set of formulas for intraclass correlation is demonstrated, and it is urged that the standard error of measurement be included when estimates of reliability are reported. In addition, three examples are provided to illustrate important concepts and familiarize the reader with the process of calculating these estimates of reliability and precision of measurement.

New measure of health‐related quality of life for patients with oropharyngeal mucositis

“…In addition, the test-retest reliabilities were good, meeting all weighted kappa and ICC coefficient requirements. [35][36][37] The symptoms scale consists of 9 items that assess various symptoms, such as pain, edema, burning, and bleeding in relation to OM. The diet scale includes 10 items that assess different degrees of dietary and eating problems.…”

“…29 The test-retest pairs for each individual item were analyzed using a weighted kappa coefficient 35,36 and the test-retest analysis of the subscale total scores was performed using intraclass correlation (ICC). 37 Values greater than 0.40 and 0.70 for weighted kappa and ICC, respectively, were accepted as satisfactory level for test-retest reliabilities. [35][36][37] Scale-level analysis was evaluated by the floor and ceiling effects.…”

“…37 Values greater than 0.40 and 0.70 for weighted kappa and ICC, respectively, were accepted as satisfactory level for test-retest reliabilities. [35][36][37] Scale-level analysis was evaluated by the floor and ceiling effects. The floor effect refers to a high percentage of subjects scoring the lowest possible score (not at all), whereas ceiling effect refer to a high percentage of subjects achieving the highest possible score (very much).…”

Effect of fatigue on force fluctuations in knee extensors in young adults

“…Cronbach's alpha, as well as the standard error of measurement (s.e.m. ), was calculated as measures of the absolute reliability (Bravo & Potvin 1991;Deneger & Ball 1993) of force fluctuation measurements between sessions, with sessions treated as a random effect (Eliasziw et al 1994). ICC values are defined as the ratio between the true variance (difference between the total and error variances) and the total variance.…”

“…In this study, the intrasession reliability was determined using the ICC(2,1) model to assess the individual repetitions within a session, and the intersession reliability was calculated using the ICC(2,3) model to examine the mean of session repetitions (Shrout & Fleiss 1979). Thus, for both intra-and intersession ICC calculations, a two-way random effects analysis of variance (ANOVA) was used, with both participants and sessions treated as random effects and the total error variance split into intersession variance and residual error variance (Shrout & Fleiss 1979;Deneger & Ball 1993). ICC values range from 0 (no reliability) to 1 (perfect reliability), with an ICC less than 0.4 rated as poor, 0.4 to less than 0.6 as fair, 0.6 to less than 0.75 as good, and greater than or equal to 0.75 as excellent (Bruton et al 2000).…”