Inter- and Intraobserver Variation in the Assessment of Preoperative Colostograms in Male Anorectal Malformations: An ARM-Net Consortium Survey

Abstract

Aim: Male patients with anorectal malformations (ARM) are classified according to presence and level of the recto-urinary fistula. This is traditionally established by a preoperative high-pressure distal colostogram that may be variably interpreted by different surgeons. The aim of this study was to evaluate the inter-and intraobserver variation in the assessment by pediatric surgeons of preoperative colostograms with respect to the level of the recto-urinary fistula. Materials and Methods: Sixteen pediatric surgeons from 14 European centers belonging to the ARM-Net Consortium twice scored 130 images of distal colostograms taken in sagittal projection at a median age of 66 days of life (range: 4-1,106 days). Surgeons were asked to classify the fistula in bulbar, prostatic, bladder-neck, no fistula, and "unclear anatomy" example. Their assessments were compared with the intraoperative findings (kappa) for two scoring rounds with an interval of 6 months (intraobserver variation). Agreement among the surgeons' scores (interobserver variation) was also calculated using Krippendorff's alpha. A kappa over 0.75 is considered excellent, between 0.40 and 0.75 fair to good, and below 0.40 poor. Surgeons were asked to score the images in "poor" and "good" quality and to provide their years of experience in ARM treatment. Midrio et al. Colostograms of Recto-Urinary Fistula Results: Agreement between the image-based rating of surgeons and the intraoperative findings ranges from 0.06 to 0.45 (mean 0.31). Interobserver variation is higher (Krippendorff's alpha between 0.40 and 0.45). Years of experience in ARM treatment does not seem to influence the scoring. The mean intraobserver variation between the two rounds is 0.64. Overall, the quality of the images is considered poor. Images categorized as having a good quality result in a statistically significant higher kappa (mean: 0.36 and 0.37 in the first and second round, respectively) than in the group of bad-quality images (mean: 0.25 and 0.23, respectively). Conclusions: There is poor agreement among experienced pediatric colorectal surgeons on preoperative colostograms. Techniques and analyses of images need to be improved in order to generate a homogeneous series of patients and make comparison of outcomes reliable.

Labeling male anorectal malformations: objective evaluation of radiologic imaging before surgery

“…A correct classification of the ARM is necessary to appropriately address the follow-up and management of possible complications. A uniform classification of ARM is also mandatory to make series comparable between centers ( 2 , 6 , 13 ).…”

“…Recently, Midrio et al ( 6 ), on behalf of the ARM-Net Consortium, reported a poor agreement among experienced pediatric colorectal surgeons on preoperative colostograms of males with ARM. In fact, the agreement between the image-based rating of surgeons and the intraoperative findings ranged from 0.06 to 0.45, and the interobserver variation was very high.…”

“…This investigation gives additional information about the presence and level of rectourinary fistulas. However, discordance in interpreting CTLxR and distal colostograms is reported, even among expert pediatric colorectal surgeons ( 6 ). To reach an objective evaluation of radiological imaging to assess the ARM type, we evaluated the utility and the accuracy of the systematic use of specific anatomical landmarks and signs during these assessments.…”