Interobserver and intraobserver agreement of three-dimensionally printed models for the classification of proximal humeral fractures

Abstract

Hypothesis This study aimed to examine whether three-dimensionally printed models (3D models) could improve interobserver and intraobserver agreement when classifying proximal humeral fractures (PHFs) using the Neer system. We hypothesized that 3D models would improve interobserver and intraobserver agreement compared with x-ray, two-dimensional (2D) and three-dimensional (3D) computed tomography (CT) and that agreement using 3D models would be higher for registrars than for consultants. Methods Thirty consecutive PHF images were selected from a state-wide database and classified by fourteen observers. Each imaging modality (x-ray, 2D CT, 3D CT, 3D models) was grouped and presented in a randomly allocated sequence on two separate occasions. Interobserver and intraobserver agreements were quantified with kappa values (κ), percentage agreement, and 95% confidence intervals (CIs). Results Seven orthopedic registrars and seven orthopedic consultants classified 30 fractures on one occasion (interobserver). Four registrars and three consultants additionally completed classification on a second occasion (intraobserver). Interobserver agreement was greater with 3D models than with x-ray (κ = 0.47, CI: 0.44-0.50, 66.5%, CI: 64.6-68.4% and κ = 0.29, CI: 0.26-0.31, 57.2%, CI: 55.1-59.3%, respectively), 2D CT (κ = 0.30, CI: 0.27-0.33, 57.8%, CI: 55.5-60.2%), and 3D CT (κ = 0.35, CI: 0.33-0.38, 58.8%, CI: 56.7-60.9%). Intraobserver agreement appeared higher for 3D models than for other modalities; however, results were not significant. There were no differences in interobserver or intraobserver agreement between registrars and consultants. Conclusion Three-dimensionally printed models improved interobserver agreement in the classification of PHFs using the Neer system. This has potential implications for using 3D models for surgical planning and teaching.

3D-printed Handheld Models Do Not Improve Recognition of Specific Characteristics and Patterns of Three-part and Four-part Proximal Humerus Fractures

“…This supports that 3D models are not clinically useful for classifying proximal humerus fractures but the question remained unanswered if other classifications, such as the Hertel LEGO description system, or specific fracture characteristics would improve with 3D modeling [9]. Another study found moderate agreement (kappa = 0.47) among 14 assessors, but they also simplified the Neer classification to three categories (two-part, three-part, and four-part fractures) and assessed the 3D-printed models without additional radiographs or CT images [3]. Again, the question whether 3D fracture models would be useful for characterization and assessment of other fracture classification systems was left open.…”

“…To date, one study found that agreement improved regarding the choice of treatment (nonoperative versus osteosynthesis versus arthroplasty) when proximal humerus fractures were assessed with 3Dprinted models [8]. Nonetheless, two studies showed that 3D-printed models improved agreement for the Neer and AO classification systems among both residents and attending surgeons, but did not reveal a difference between both groups [3,9]. Although they conducted valuable work, they did not account for characterization and other fracture classification systems.…”

Improving Patient Understanding of Femoroacetabular Impingement Syndrome With Three-Dimensional Models

“…Other studies have demonstrated the benefits of 3D printed models among residents, such as improving interobserver reliability of a proximal humerus fracture classification system, compared with radiographs and CT alone. 24 Education level was stratified in each cohort to investigate whether there was an intercohort difference between levels of education and reported patient understanding of their injury and surgical plan. All responses were on a continuous scale of 0 (no understanding) to 100 (expert understanding).…”

“…Other studies have demonstrated the benefits of 3D printed models among residents, such as improving interobserver reliability of a proximal humerus fracture classification system, compared with radiographs and CT alone. 24 …”

CORR Insights®: 3D-printed Handheld Models Do Not Improve Recognition of Specific Characteristics and Patterns of Three-part and Four-part Proximal Humerus Fractures

“…The reliability of assessing proximal humerus fracture patterns using widely-held classification systems such as that of Neer and Hertel based on plain radiographs has been fairly low, though the addition of advanced imaging such as two-dimensional (2-D) and three-dimensional (3-D) CT scans appears to improve both inter- and intraobserver reliability [4]. More recently, using 3D printed models alone in the surgical planning process has been found to improve interobserver reliability over plain radiographs and both 2D and 3D CT scans using the Neer system, though the observed agreement with the printed models was only moderate [2]. In these studies, where the kappa values using the guidelines of Landis and Koch were reported as “substantial” and “moderate,” respectively, we must recognize that a high proportion of cases will still be misclassified.…”