AutoEncoder-Driven Multimodal Collaborative Learning for Medical Image Synthesis

Cao, Bing; Bi, Ziqiang; Hu, Qinghua; Zhang, Han; Wang, Nannan; Gao, Xinbo; Shen, Dinggang

doi:10.1007/s11263-023-01791-0

Cited by 11 publications

(2 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Modification to network architectures may include the exploration of variational U-Nets [46,67,68] generative adversarial networks (GANs) [69,70], variational autoencoders [71,72], transformer-based models [73], and other state-of-the-art methods. Additional network modifications may include the incorporation of a contrastive loss term [74] and data augmentation techniques [75].…”

Section: Future Directionmentioning

confidence: 99%

Synthetic Knee MRI T1p Maps as an Avenue for Clinical Translation of Quantitative Osteoarthritis Biomarkers

Tong,

Tolpadi,

Bhattacharjee

et al. 2023

Bioengineering

View full text Add to dashboard Cite

A 2D U-Net was trained to generate synthetic T1p maps from T2 maps for knee MRI to explore the feasibility of domain adaptation for enriching existing datasets and enabling rapid, reliable image reconstruction. The network was developed using 509 healthy contralateral and injured ipsilateral knee images from patients with ACL injuries and reconstruction surgeries acquired across three institutions. Network generalizability was evaluated on 343 knees acquired in a clinical setting and 46 knees from simultaneous bilateral acquisition in a research setting. The deep neural network synthesized high-fidelity reconstructions of T1p maps, preserving textures and local T1p elevation patterns in cartilage with a normalized mean square error of 2.4% and Pearson’s correlation coefficient of 0.93. Analysis of reconstructed T1p maps within cartilage compartments revealed minimal bias (−0.10 ms), tight limits of agreement, and quantification error (5.7%) below the threshold for clinically significant change (6.42%) associated with osteoarthritis. In an out-of-distribution external test set, synthetic maps preserved T1p textures, but exhibited increased bias and wider limits of agreement. This study demonstrates the capability of image synthesis to reduce acquisition time, derive meaningful information from existing datasets, and suggest a pathway for standardizing T1p as a quantitative biomarker for osteoarthritis.

show abstract

Section: Future Directionmentioning

confidence: 99%

Synthetic Knee MRI T1p Maps as an Avenue for Clinical Translation of Quantitative Osteoarthritis Biomarkers

Tong,

Tolpadi,

Bhattacharjee

et al. 2023

Bioengineering

View full text Add to dashboard Cite

show abstract

“…The imputation of missing protein markers is a missing data imputation problem, and a number of AI-based methods, particularly deep learning methods, have been developed for missing data imputation in various domains (18–22). However, current literature on image imputation in medical imaging primarily focuses on radiology datasets (20–22), with limited research exploring the potential of deep learning models for marker synthesis in multiplex images (23, 24).…”

Section: Introductionmentioning

confidence: 99%

Deep Learning Model Imputes Missing Stains in Multiplex Images

Shaban,

Lassoued,

Canubas

et al. 2023

Preprint

View full text Add to dashboard Cite

Multiplex staining enables simultaneous detection of multiple protein markers within a tissue sample. However, the increased marker count increased the likelihood of staining and imaging failure, leading to higher resource usage in multiplex staining and imaging. We address this by proposing a deep learning-basedMArker imputation model for multipleXIMages (MAXIM) that accurately impute protein markers by leveraging latent biological relationships between markers. The model’s imputation ability is extensively evaluated at pixel and cell levels across various cancer types. Additionally, we present a comparison between imputed and actual marker images within the context of a downstream cell classification task. The MAXIM model’s interpretability is enhanced by gaining insights into the contribution of individual markers in the imputation process. In practice, MAXIM can reduce the cost and time of multiplex staining and image acquisition by accurately imputing protein markers affected by staining issues.

show abstract