Rationale There is pathologic evidence that suggests the current outbreak of progressive massive fibrosis (PMF) among U.S. coal miners is related to excessive exposure to silica. Type of PMF (silicotic, coal, mixed, or indeterminate) is determined histopathologically, although there are currently no standardized case definitions or histologic scoring systems. We developed standardized tools and evaluated both intra-and inter-rater agreement of expert pathologists' classifications using autopsy specimens from U.S. coal miners with PMF. Methods: Seven occupational pathologists and four pulmonologists created and pilot-tested standardized case definitions and a scoring system for PMF type. The pathologists then reviewed de-identified scanned bright-field microscopy images of H&E stained lung sections from 462 deceased U.S. coal miners with PMF taken from the National Institute for Occupational Safety and Health's National Coal Workers' Autopsy Study (NCWAS) from 1970 to 2013. Four raters chose to read in pairs, yielding five independent reads. The pathologists were blinded to the miners' clinical information. Type of PMF was categorized as (1) silicotic (>75% silicotic nodules); (2) mixed (>25 to ≤ 75% silicotic nodules);(3) coal (≤25% silicotic nodules); or (4) indeterminate. We combined the silicotic and mixed categories for analysis. Simple percent agreement as well as simple and weighted Cohen's kappa coefficients (κ) were used to evaluate intra-and inter-rater reliability. Results Intra-rater κ ranged from 0.63 to 0.83, indicating substantial or almost perfect agreement for all raters (Table 1). On average, the pathologists categorized PMF cases as the same PMF type 88% of the time (range of percent agreement 83 -93%). Inter-rater agreement ranged from fair to substantial agreement (κ range 0.31 -0.68) in pairwise comparisons (Table 1). An overall inter-rater weighted Cohen's kappa for all readers indicated substantial agreement (κ = 0.62). Conclusions Intrarater reliability for type of PMF was substantial among five independent pathologic readings of deceased U.S. coal miners with PMF. Overall inter-rater reliability was also substantial, with individual comparisons of pathologists ranging from fair to substantial agreement. We would expect agreement to improve with repeated training sessions and the use of standard images for these classifications. These tools may hold promise for the development of standardized histologic scoring approaches to PMF in coal miners, similar to the International Labour Organization standard images for the interpretation of chest radiographs for pneumoconiosis.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.