BackgroundThoracic ultrasonography (TUS) can be used to assess the extent and severity of lung lesions associated with bronchopneumonia (BP) in feedlot cattle.Hypothesis/ObjectivesTo assess inter‐rater agreement and reliability of TUS findings in feedlot cattle, with or without naturally occurring BP.AnimalsFeedlot steers with (n = 210) or without (n = 107) clinical signs of BP that were assessed by TUS in a previous case‐control study.MethodsA random sample of 50 TUS videos (16‐s duration) were scored by 6 raters with various levels of TUS expertise. Lung consolidation, comet tail artifacts, pleural irregularity and effusion were scored. Inter‐rater agreement was assessed using raw percentage of agreement (Pa), Cohen's and Fleiss’ Kappa (κ), and Gwet agreement coefficient (AC1). Intra‐class correlation (ICC) was determined for variables with continuous measurements (mixed factorial design).ResultsMedian (interquartile range [IQR]) Pa were 0.84 (0.80‐0.89), 0.82 (0.80‐0.87), 0.62 (0.53‐0.67), and 0.82 (0.75‐0.86) for presence of lung consolidation, comet tails, pleural irregularity, and pleural effusion, respectively. For the same lesions, Fleiss κ (95% confidence intervals [CI]) were 0.67 (0.49‐0.86), 0.56 (0.33‐0.80), 0.20 (−0.05 to 0.44), and 0.36 (0.10‐0.61), respectively. AC1 were 0.68 (0.51‐0.86), 0.73 (0.58‐0.89), 0.21 (−0.01 to 0.44), and 0.71 (0.51‐0.92), respectively. Moderate reliability was found among raters for all quantitative variables (ICC ranged from 0.52 to 0.70).Conclusions and Clinical ImportanceInter‐rater agreement was good for presence of lung consolidation, comet tails and pleural effusion (based on Pa and AC1) but was slight to poor for pleural irregularity.