CheXRelNet: An Anatomy-Aware Model for Tracking Longitudinal Relationships Between Chest X-Rays

Abstract

Despite the progress in utilizing deep learning to automate chest radiograph interpretation and disease diagnosis tasks, change between sequential Chest X-rays (CXRs) has received limited attention. Monitoring the progression of pathologies that are visualized through chest imaging poses several challenges in anatomical motion estimation and image registration, i.e., spatially aligning the two images and modeling temporal dynamics in change detection. In this work, we propose CheXRelNet, a neural model that can track longitudinal pathology change relations between two CXRs. CheXRelNet incorporates local and global visual features, utilizes inter-image and intra-image anatomical information, and learns dependencies between anatomical region attributes, to accurately predict disease change for a pair of CXRs. Experimental results on the Chest ImaGenome dataset show increased downstream performance compared to baselines. Code is available at https://github.com/PLAN-Lab/ChexRelNet.

Employing similarity to highlight differences: On the impact of anatomical assumptions in chest X-ray registration methods

“…In [22], progression of lung infiltrates is predicted in several zones, by taking the previous images and neighboring patches into account via a Gated Recurrent Unit (GRU). Global and local dependencies between anatomical regions are the focus of [20], who employ a graph attention network.…”

“…Alternative technical approaches in medical image analysis for assisting with this problem have been suggested, like geometric correlation maps [35], comparison via Siamese networks [26], Gated Recurrent Units (GRUs) [22] or graph attention [20], but the most commonly used technique remains (deformable) image registration. Originally formulated as an optimization problem with high computational costs, learning-based optimization through neural networks has become increasingly popular since the invention of spatial transformers [19].…”

Controllable Chest X-Ray Report Generation from Longitudinal Representations

“…Bannur et al (2023) introduce a self-supervised multimodal approach that models longitudinal CXRs from image-level features as a joint temporal representation to better align text and image. Karwande et al (2022) have proposed an anatomy-aware approach to classifying if a finding has improved or worsened by modelling longitudinal representations between CXRs with graph attention networks (Veličković et al, 2018). Similar to Karwande et al (2022), we project longitudinal studies into a joint representation based on anatomical representations rather than image-level features.…”

“…Karwande et al (2022) have proposed an anatomy-aware approach to classifying if a finding has improved or worsened by modelling longitudinal representations between CXRs with graph attention networks (Veličković et al, 2018). Similar to Karwande et al (2022), we project longitudinal studies into a joint representation based on anatomical representations rather than image-level features. However, we extract the anatomical representations from Faster R-CNN (Ren et al, 2015), as in Dalla Serra et al (2023).…”