Ultrasound grading of thyroid nodules using the BTA U-scoring guidelines – Is there evidence of intra-and interobserver variability?

Abstract

Introduction U-score ultrasound classification (graded U1-U5) is widely used to grade thyroid nodules based on benign and malignant sonographic features. It is well established that ultrasound is an operator-dependent imaging modality and thus more susceptible to subjective variances between operators when using imaging-based scoring systems. We aimed to assess whether there is any intra- or interobserver variability when U-scoring thyroid nodules and whether previous thyroid ultrasound experience has an effect on this variability. Methods A total of 14 ultrasound operators were identified (five experienced thyroid operators, five with intermediate experience and four with no experience) and were asked to U-score images from 20 thyroid cases shown as a single projection, with and without Doppler flow. The cases were subsequently rescored by the 14 operators after six weeks. The first and second round U-scores for the three operator groups were then analysed using Fleiss’ kappa to assess interobserver variability and Cochran’s Q test to determine any intraobserver variability. Results We found no significant interobserver variability on combined assessment of all operators with fair agreement in round 1 (Fleiss’ kappa = 0.30, p <0.0001) and slight agreement in round 2 (Fleiss’ kappa = 0.19, p < 0.0001). Cochran’s Q test revealed no significant intraobserver variability in all 14 operators between round 1 and round 2 (all p>0.05). Conclusions We found no statistically significant inter- or intraobserver variability in the U-scoring of thyroid nodules between all participants reinforcing the validity of this scoring method in clinical practice, allaying concerns regarding potential subjective biases in reporting.

Evaluation of the departmental inter-rater reliability when scoring thyroid nodules according to the British Thyroid Association Ultrasound-classification model: Is there significant disagreement?

“…Although the level of agreement for the U-categories was only fair ( K = 0.21) in the department, this appears in line with the study conducted by Couzins et al 12 where the agreement for the experienced group was of K = 0.24–0.34. This, however, is significantly lower from the one reported by Weller et al 11 ( K = 0.73).…”

“…Couzins et al, 12 on the contrary, studied the inter- and intra-operator reliability on 20 TNs between 14 US operators with different experience. Although no statistically significant variability existed when U-scoring, the study demonstrated only slight to fair agreement between all the participants and within the experienced group only ( K = 0.19–0.34).…”