A Multimodal Search Engine for Medical Imaging Studies

Pinho, Eduardo; Godinho, Tiago Marques; Valente, Frederico; Costa, Carlos

doi:10.1007/s10278-016-9903-z

Cited by 19 publications

(14 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The current work focuses on X-ray medical images, whereas it is common to use different imaging modalities (such as X-ray, CT, PET, MRI, angiography, and sonography) to provide complementary information about a special patient. The multimodality medical image retrieval has potential applications in diagnosis, training, and research [46]. In future work, we intend to extend the multi-texton representation approach on the multimodality medical image repositories containing a diverse collection of data, achieving higher CBIR performance utilizing complementary information from different modalities.…”

Section: Resultsmentioning

confidence: 99%

Medical Image Retrieval Using Multi-Texton Assignment

2017

View full text Add to dashboard Cite

In this paper, we present a multi-texton representation method for medical image retrieval, which utilizes the locality constraint to encode each filter bank response within its local-coordinate system consisting of the k nearest neighbors in texton dictionary and subsequently employs spatial pyramid matching technique to implement feature vector representation. Comparison with the traditional nearest neighbor assignment followed by texton histogram statistics method, our strategies reduce the quantization errors in mapping process and add information about the spatial layout of texton distributions and, thus, increase the descriptive power of the image representation. We investigate the effects of different parameters on system performance in order to choose the appropriate ones for our datasets and carry out experiments on the IRMA-2009 medical collection and the mammographic patch dataset. The extensive experimental results demonstrate that the proposed method has superior performance.

show abstract

Section: Resultsmentioning

confidence: 99%

Medical Image Retrieval Using Multi-Texton Assignment

2017

View full text Add to dashboard Cite

show abstract

“…Pinho et al designed a novel biomedical search engine. An extensible model for biomedical images combined with an open-source picture archiving and communication model with profile-based capabilities has been utilized [ 22 ]. Long designed a novel search engine model for a supplemental health applications.…”

Section: Literature Reviewmentioning

confidence: 99%

Deep learning based search engine for biomedical images using convolutional neural networks

Mishra

Tripathi

2021

Multimed Tools Appl

View full text Add to dashboard Cite

The development of efficient search engine queries for biomedical images, especially in case of query-mismatch is still defined as an ill-posed problem. Vector-space model is found to be useful for handling the query-mismatch issue. However, vector-space model does not consider the relational details among the keywords and biomedical image search space is not evaluated. Therefore, in this paper, we have proposed a deep learning based fusion vectorspace based model. The proposed model enhances the biomedical image query similarity matching approach by fusing the vector space model and convolutional neural networks. Deep learning model is defined by converting the vector-space model to a classification model. Finally, deep learning model is trained to implement the search engine for biomedical images. Extensive experiments reveal that the proposed model achieves significant improvement over the existing models.

show abstract

“…In audiovisual speech recognition, the main goal is to improve speech recognition performance by combining the visual information with audio/speech signals [13], [14]. Content analysis such as automatic shot-boundary detection, multimedia event detection, searching visual and multimodal content in a dataset are few examples of multimedia content indexing and retrieval [15], [16], [17], [18]. Human-robot collaboration, human emotion recognition, human-computer interaction, and automatic assessment of depression and stress comes under the category of understanding human behaviour from multimodal input data during social interactions [19], [20], [21], [22], [23].…”

Section: A Multimodal Learningmentioning

confidence: 99%

Subspace Support Vector Data Description

Sohrab

Raitoharju

Gabbouj

et al. 2018

2018 24th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

In this paper, we propose a novel method for projecting data from multiple modalities to a new subspace optimized for one-class classification. The proposed method iteratively transforms the data from the original feature space of each modality to a new common feature space along with finding a joint compact description of data coming from all the modalities. For data in each modality, we define a separate transformation to map the data from the corresponding feature space to the new optimized subspace by exploiting the available information from the class of interest only. The data description in the new subspace is obtained by Support Vector Data Description. We also propose different regularization strategies for the proposed method and provide both linear and non-linear formulation. We conduct experiments on two multimodal datasets and compare the proposed approach with baseline and recently proposed one-class classification methods combined with early fusion and also considering each modality separately. We show that the proposed Multimodal Subspace Support Vector Data Description outperforms all the methods using data from a single modality and performs better or equally well than the methods fusing data from all modalities.

show abstract

A Multimodal Search Engine for Medical Imaging Studies

Cited by 19 publications

References 23 publications

Medical Image Retrieval Using Multi-Texton Assignment

Medical Image Retrieval Using Multi-Texton Assignment

Deep learning based search engine for biomedical images using convolutional neural networks

Subspace Support Vector Data Description

Contact Info

Product

Resources

About