Objectives We hypothesized that published performances of algorithms for artificial intelligence (AI) pneumothorax (PTX) detection in chest radiographs (CXRs) do not sufficiently consider the influence of PTX size and confounding effects caused by thoracic tubes (TTs). Therefore, we established a radiologically annotated benchmarking cohort (n = 6446) allowing for a detailed subgroup analysis. Materials and Methods We retrospectively identified 6434 supine CXRs, among them 1652 PTX-positive cases and 4782 PTX-negative cases. Supine CXRs were radiologically annotated for PTX size, PTX location, and inserted TTs. The diagnostic performances of 2 AI algorithms (“AI_CheXNet” [Rajpurkar et al], “AI_1.5” [Guendel et al]), both trained on publicly available datasets with labels obtained from automatic report interpretation, were quantified. The algorithms' discriminative power for PTX detection was quantified by the area under the receiver operating characteristics (AUROC), and significance analysis was based on the corresponding 95% confidence interval. A detailed subgroup analysis was performed to quantify the influence of PTX size and the confounding effects caused by inserted TTs. Results Algorithm performance was quantified as follows: overall performance with AUROCs of 0.704 (AI_1.5) / 0.765 (AI_CheXNet) for unilateral PTXs, AUROCs of 0.666 (AI_1.5) / 0.722 (AI_CheXNet) for unilateral PTXs smaller than 1 cm, and AUROCs of 0.735 (AI_1.5) / 0.818 (AI_CheXNet) for unilateral PTXs larger than 2 cm. Subgroup analysis identified TTs to be strong confounders that significantly influence algorithm performance: Discriminative power is completely eliminated by analyzing PTX-positive cases without TTs referenced to control PTX-negative cases with inserted TTs. Contrarily, AUROCs increased up to 0.875 (AI_CheXNet) for large PTX-positive cases with inserted TTs referenced to control cases without TTs. Conclusions Our detailed subgroup analysis demonstrated that the performance of established AI algorithms for PTX detection trained on public datasets strongly depends on PTX size and is significantly biased by confounding image features, such as inserted TTS. Our established, clinically relevant and radiologically annotated benchmarking cohort might be of great benefit for ongoing algorithm development.

Objective: Alzheimer disease (AD) is characterized by amyloid β (Aβ) plaques and neurofibrillary tau tangles, but increasing evidence suggests that neuroinflammation also plays a key role, driven by the activation of microglia. Aβ and tau pathology appear to spread along pathways of highly connected brain regions, but it remains elusive whether microglial activation follows a similar distribution pattern. Here, we assess whether connectivity is associated with microglia activation patterns. Methods: We included 32 Aβ-positive early AD subjects (18 women, 14 men) and 18 Aβ-negative age-matched healthy controls (10 women, 8 men) from the prospective ActiGliA (Activity of Cerebral Networks, Amyloid and Microglia in Aging and Alzheimer's Disease) study. All participants underwent microglial activation positron emission tomography (PET) with the third-generation mitochondrial 18 kDa translocator protein (TSPO) ligand [ 18 F]GE-180 and magnetic resonance imaging (MRI) to measure resting-state functional and structural connectivity. Results: We found that inter-regional covariance in TSPO-PET and standardized uptake value ratio was preferentially distributed along functionally highly connected brain regions, with MRI structural connectivity showing a weaker

Objectives Diagnostic accuracy of artificial intelligence (AI) pneumothorax (PTX) detection in chest radiographs (CXR) is limited by the noisy annotation quality of public training data and confounding thoracic tubes (TT). We hypothesize that in-image annotations of the dehiscent visceral pleura for algorithm training boosts algorithm’s performance and suppresses confounders. Methods Our single-center evaluation cohort of 3062 supine CXRs includes 760 PTX-positive cases with radiological annotations of PTX size and inserted TTs. Three step-by-step improved algorithms (differing in algorithm architecture, training data from public datasets/clinical sites, and in-image annotations included in algorithm training) were characterized by area under the receiver operating characteristics (AUROC) in detailed subgroup analyses and referenced to the well-established “CheXNet” algorithm. Results Performances of established algorithms exclusively trained on publicly available data without in-image annotations are limited to AUROCs of 0.778 and strongly biased towards TTs that can completely eliminate algorithm’s discriminative power in individual subgroups. Contrarily, our final “algorithm 2” which was trained on a lower number of images but additionally with in-image annotations of the dehiscent pleura achieved an overall AUROC of 0.877 for unilateral PTX detection with a significantly reduced TT-related confounding bias. Conclusions We demonstrated strong limitations of an established PTX-detecting AI algorithm that can be significantly reduced by designing an AI system capable of learning to both classify and localize PTX. Our results are aimed at drawing attention to the necessity of high-quality in-image localization in training data to reduce the risks of unintentionally biasing the training process of pathology-detecting AI algorithms. Key Points • Established pneumothorax-detecting artificial intelligence algorithms trained on public training data are strongly limited and biased by confounding thoracic tubes. • We used high-quality in-image annotated training data to effectively boost algorithm performance and suppress the impact of confounding thoracic tubes. • Based on our results, we hypothesize that even hidden confounders might be effectively addressed by in-image annotations of pathology-related image features.

Artificial intelligence (AI) algorithms evaluating [supine] chest radiographs ([S]CXRs) have remarkably increased in number recently. Since training and validation are often performed on subsets of the same overall dataset, external validation is mandatory to reproduce results and reveal potential training errors. We applied a multicohort benchmarking to the publicly accessible (S)CXR analyzing AI algorithm CheXNet, comprising three clinically relevant study cohorts which differ in patient positioning ([S]CXRs), the applied reference standards (CT-/[S]CXR-based) and the possibility to also compare algorithm classification with different medical experts’ reading performance. The study cohorts include [1] a cohort, characterized by 563 CXRs acquired in the emergency unit that were evaluated by 9 readers (radiologists and non-radiologists) in terms of 4 common pathologies, [2] a collection of 6,248 SCXRs annotated by radiologists in terms of pneumothorax presence, its size and presence of inserted thoracic tube material which allowed for subgroup and confounding bias analysis and [3] a cohort consisting of 166 patients with SCXRs that were evaluated by radiologists for underlying causes of basal lung opacities, all of those cases having been correlated to a timely acquired computed tomography scan (SCXR and CT within < 90 min). CheXNet non-significantly exceeded the radiology resident (RR) consensus in the detection of suspicious lung nodules (cohort [1], AUC AI/RR: 0.851/0.839, p = 0.793) and the radiological readers in the detection of basal pneumonia (cohort [3], AUC AI/reader consensus: 0.825/0.782, p = 0.390) and basal pleural effusion (cohort [3], AUC AI/reader consensus: 0.762/0.710, p = 0.336) in SCXR, partly with AUC values higher than originally published (“Nodule”: 0.780, “Infiltration”: 0.735, “Effusion”: 0.864). The classifier “Infiltration” turned out to be very dependent on patient positioning (best in CXR, worst in SCXR). The pneumothorax SCXR cohort [2] revealed poor algorithm performance in CXRs without inserted thoracic material and in the detection of small pneumothoraces, which can be explained by a known systematic confounding error in the algorithm training process. The benefit of clinically relevant external validation is demonstrated by the differences in algorithm performance as compared to the original publication. Our multi-cohort benchmarking finally enables the consideration of confounders, different reference standards and patient positioning as well as the AI performance comparison with differentially qualified medical readers.

Background: Alzheimer′s disease (AD) is characterized by amyloid-β; (Aβ) plaques, neurofibrillary tau tangles and neuroinflammation leading to brain functional connectivity changes and cognitive decline. There is evidence, that microglial activity is increased in AD and cognitive decline. Aβ and tau pathology appear to spread along pathways of highly connected brain regions, but it remains elusive if microglial activation follows a similar distribution pattern. Methods: Thirty-two early AD subjects and 18 age-matched healthy cognitively normal controls were included from the prospective ActiGliA study. Differences between the diagnostic groups were explored for translocator protein (TSPO) positron emission tomography (PET) microglial activation, diffusion tensor imaging (DTI) structural connectivity and magnetic resonance imaging (MRI) functional connectivity. Associations between PET microglial activation with cognitive impairment, dementia severity and MRI connectivity measures were investigated within the diagnostic groups. Results: AD patients showed increased TSPO PET tracer uptake bilaterally in the parahippocampal region compared to cognitively normal controls. Higher TSPO PET was associated with cognitive impairment and dementia severity in a disease stage dependent fashion. Inter-regional covariance in TSPO PET and standardized uptake value ratio (SUVR) was found to be preferentially distributed along functionally highly connected brain regions, with MRI structural connectivity showing a weaker association with microglial activation. Conclusion: Neuroinflammation in AD is associated with clinical disease presentation, and like tau pathology, microglial activation seems to spread preferentially along highly connected brain regions. These findings support the important role of microglia in neurodegeneration and suggest that disease spreading throughout the brain along vulnerable connectivity pathways could guide future interventional anti-inflammatory therapy approaches to prevent disease progression.