DeepImageTranslator V2: analysis of multimodal medical images using semantic segmentation maps generated through deep learning

Abstract

Introduction: Analysis of multimodal medical images often requires the selection of one or many anatomical regions of interest (ROIs) for extraction of useful statistics. This task can prove laborious when a manual approach is used. We have previously developed a user-friendly software tool for image-to-image translation using deep learning. Therefore, we present herein an update to the DeepImageTranslator software with the addiction of a tool for multimodal medical image segmentation analysis (hereby referred to as the MMMISA). Methods: The MMMISA was implemented using the Tkinter library. Backend computations were implemented using the Pydicom, Numpy, and OpenCV libraries. We tested our software using 4188 whole-body axial 2-deoxy-2-[18F]-fluoroglucose-position emission tomography/computed tomography ([18F]-FDG-PET/CT) slices of 10 patients from the ACRIN-HNSCC (American College of Radiology Imaging Network-Head and Neck Squamous Cell Carcinoma) database. Using the deep learning software DeepImageTranslator, a model was trained with 36 randomly selected CT slices and manually labelled semantic segmentation maps. Utilizing the trained model, all the CT scans of the 10 HNSCC patients were segmented with high accuracy. Segmentation maps generated using the deep convolutional network were then used to measure organ specific [18F]-FDG uptake. We also compared measurements performed using the MMMISA and those made with manually selected ROIs. Results: The MMMISA is a tool that allows user to select ROIs based on deep learning-generated segmentation maps and to compute accurate statistics for these ROIs based on coregistered multimodal images. We found that organ-specific [18F]-FDG uptake measured using multiple manually selected ROIs is concordant with whole-tissue measurements made with segmentation maps using the MMMISA tool.

Applications of Vision Transformers in Retinal Imaging: A Systematic Review

“…In the field of radiology, deep learning models have been used to analyze medical images such as X-rays, CT scans, and MRIs for the diagnosis and treatment of various conditions [23][24][25]. ViTs have also been explored for use in natural language processing tasks, such as extracting information from electronic health records and detecting adverse drug events [26][27][28].…”

Impact of multi-source data augmentation on performance of convolutional neural networks for abnormality classification in mammography

“…Since the DDSM mammograms are in LJPEG formats, the Stanford PVRG JPEG codec v1.1 was employed to read DDSM images and convert them into 16-bit grayscale PNG images (13). CBIS-DDSM and EMBED images are in DICOM format and were converted into 16-bit grayscale PNG files (22)(23)(24)(25). All images were rescaled to 800 × 600 with bicubic interpolation and anti-aliasing to make them fit into 8 GB memory Graphic Processing Units (GPUs) for improved reproducibility.…”

Identification of early diagnostic biomarkers for breast cancer through bioinformatics analysis

“…Moreover, the work by Saeed Roshani et al [8] implies that sensor technology could be a potent tool for diagnosing early-stage breast cancer. En Zhou Ye et al, [9] found that employing deep learning-generated semantic segmentation mapping analysis of multimodal medical imagery could aid in diagnosing malignant breast tissue. Although mammography is considered the most efficacious diagnostic tool for breast cancer, it is not devoid of risks such as false positives, radiation exposure, and discomfort associated with the procedure.…”