A comparison of the reliability and validity of bone stock loss classification systems used for revision hip surgery

Classifications In Brief: Paprosky Classification of Acetabular Bone Loss

“…Intraobserver j values for the expert group were poor to moderate (j = 0.14 and 0.33 for two separate reviewers in one study [4], and average j = 0.46 [0.27-0.60] for a cohort of three reviewers in another study [1]), whereas the nonexpert group was fair to moderate (j = 0.31 and 0.41 for two separate reviewers in one study [4], and average j = 0.37 (0.26-0.50) for a cohort of three reviewers in another study [1]), unexpectedly showing greater variability in the experts' analysis and no substantial improvement over the nonexperts' evaluation. Interobserver reliability was similarly inconsistent; expert j values were substantially improved compared with nonexperts in one study (expert, j = 0.56; nonexpert, j = 0.27) [4], and approximately equivalent in another study (experts, j = 0.25 and 0.27; nonexperts, j = 0.18 and 0.31) [1].…”

“…The kappa (j) analysis is a representation of the proportion of agreement beyond that expected by chance [10]; a value of zero indicates no agreement better than chance and a value of 1 indicates agreement between two scorers is perfect [10]. Intraobserver reliability for the Paprosky classification has varied with reported j values ranging from 0.14 to 0.75 [1,4,12,19]. Most of the time the values were between 0.3 and 0.6, indicating poor to good agreement.…”

“…Most of the time the values were between 0.3 and 0.6, indicating poor to good agreement. Interobserver reliability has shown even greater variability with reported j values ranging from 0.02 to 0.79 [1,4,12,19] with the majority of values between 0.4 and 0.6, suggesting anything from good agreement to essentially no agreement better than that is the result of chance. The source of this heterogeneity is likely multifactorial and includes the loosely defined and relatively subjective categories of mild, moderate, and severe used by the Paprosky system, the partial obstruction of acetabular features by the radiopaque cup on routine radiographs, and potential discrepancies in the measurement of hip center migration between reviewers which is a major differentiating feature between Types 1 and 2 defects.…”

“…Two studies have compared reliability between designated ''expert'' cohorts, comprised of practicing orthopaedic surgeons with an arthroplasty focus, and ''nonexpert'' cohorts, consisting of senior level orthopaedic residents or their British equivalents [1,4]. Intraobserver j values for the expert group were poor to moderate (j = 0.14 and 0.33 for two separate reviewers in one study [4], and average j = 0.46 [0.27-0.60] for a cohort of three reviewers in another study [1]), whereas the nonexpert group was fair to moderate (j = 0.31 and 0.41 for two separate reviewers in one study [4], and average j = 0.37 (0.26-0.50) for a cohort of three reviewers in another study [1]), unexpectedly showing greater variability in the experts' analysis and no substantial improvement over the nonexperts' evaluation.…”

“…Paprosky et al [11], in their original study describing the classification, indicated that 100% of Type 1, 89% of Type 2, and 95% of Type 3 defects classified preoperatively were found to accurately reflect findings at the time of surgery. In another study, Gozzard et al [4] reported good validity (j = 0.65) when comparing defects on preoperative radiographs with intraoperative findings; however, the exact method used to describe intraoperative defects was not characterized.…”