Comparing tangible retinal image characteristics with deep learning features reveals their complementarity for gene association and disease prediction

Abstract

Advances in computer-aided analyses, including deep learning (DL), are transforming medical imaging by enabling automated disease risk predictions and aiding clinical interpretation. However, DL’s outputs and latent variables (LVs) often lack interpretability, impeding clinical trust and biological insight. In this study, we evaluatedRETFound, a foundation model for retinal images, using a dataset annotated with clinically interpretable tangible image features (TIFs). Our findings revealed that individual LVs poorly represent complex TIFs but achieve higher accuracy when combined linearly. Fine-tuningRETFoundto predict TIFs, providing “deep TIFs” provided better, but far from perfect surrogates, highlighting the limitations of DL approaches to fully characterise retinal images. Yet, our genetic analyses showed that deep TIFs exhibit heritability comparable to or exceeding measured TIFs but highlighted non-genetic variability in LVs. While measured and deep TIFs, as well as LVs, showed overlapping genetic and disease associations, their complementarity enhances prediction models. Notably, deep TIFs excelled in ocular disease prediction, emphasising their potential to refine retinal diagnostics and bridge gaps in conventional assessments of vascular morphology.