Multimodal, Multianatomical and Multidimensional Medical Image Retrieval System

Abstract

Recently, there is a rapid use of digital imaging information in healthcare enterprises. Hence, it becomes laborious to manage and query in such large databases which need effi- cient medical image retrieval systems. Also, the multi-modal and multi-dimensional aspects of medical images make this a much more demanding task. The imaging data such as the CT, and MRI from the scanners is in the form of 3D images which consist of several slices stacked upon each other. While medical images such as the X- rays are in the 2D format. This imbalance in the medical image databases leads to de- velop an integrated 2D and 3D medical image retrieval sys- tem using Deep Learning architectures. In this context, an integrated framework with hybrid architectures consisting of convolutional neural networks and autoencoder is proposed. A heterogeneous database comprises of 2D and 3D images produced from different sources of modalities to train the proposed networks is used. The learned features are used to retrieve the medical images. Five unsupervised CNN mod- els namely LeNetCoder, VGGCoder, Noisy VGGCoder, LSTM VGGCoder, and ResCoder were trained and tested for both the 2D and 3D images. Finally, the performances of all the models are compared with the metrics like Precision, Recall, and F-Score. Among them, ResCoder has the highest mean average precision (MAP) of 0.96 for 2D and 0.92 for 3D im- ages in this framework.