Using content‐based image retrieval of dermoscopic images for interpretation and education: A pilot study

Abstract

Recent advances in artificial intelligence (AI) and computer-aided decision support methods have produced various efficient ways to allow for learning about skin problems. 1 In particular, advances in machine learning have spurred novel retrieval algorithms and aroused interest in content-based image retrieval (CBIR) techniques, where computer vision methods are applied to search for similar images to a "query" image based on the content of the image and visual clues such as color, shape, and pattern, from large databases. 2 In the medical domain, CBIR is designed to assist with finding similar, labeled, medical images from a curated database. Within the dermatology context, CBIR can assist with diagnosis or education by comparing visually similar skin lesion images, 3 removing the difficulties that can arise when trying to describe images with words. Since the database and the algorithms for these systems are curated for a specific area or problem, users are less likely to encounter irrelevant images, one of the main problems with generic search engines. Despite the proposed benefits of modern CBIR systems, most CBIR-related research to date has focused on improving the accuracy of AI systems for diagnostic decisions 4,5 : we know little about the perceived utility and usability of CBIR systems for end users from a human-computer interaction (HCI) perspective. 6 In this paper, we describe a pilot study on how an interactive dermoscopic

Implementing Relevance Feedback for Content-Based Medical Image Retrieval

“…Scientists have developed support systems such as the computer-aided diagnosis (CAD) system and content-based medical image retrieval (CBMIR) system to help radiologists interpret medical images. The most important benefit of CBMIR is that it aids radiologists in identifying similar medical images in recalling previous cases during diagnosis [5]- [7]. Many content-based medical image retrieval (CBMIR) systems are based on image similarity, whereby a user enters a query image, and the system responds by supplying the most similar image focused on a certain measure of similarity.…”

Explainable artificial intelligence in skin cancer recognition: A systematic review

“…2/14 studies evaluated whether models had learned human-interpretable features and found that they had indeed learned to detect relevant structures [48,49]. 3/14 studies (discussed in detail in Section 2.2) investigated humanemachine interaction in the context of XAI [17,50,51].…”

“…3/13 studies [17,50,51] investigated the influence of XAI, more precisely of two CBIR systems, on the predictive accuracy and confidence of human users: Sadeghi et al [51] evaluated the framework introduced by Tschandl et al [16] with regards to whether it improved the classification performance of 16 non-medical graduate students on a 4-class classification task (basal cell carcinoma, melanoma, nevus and seborrheic keratosis). In a more recent study, Sadgehi et al [50] analyse the predictive performance of 14 non-medical graduate students and additionally evaluate whether the students perceive the system as useful for educational purposes. Both studies find improvements in the predictive performance of the graduate students when assisted by the CBIR: from on average 10.31/20 (51.56%) correct predictions to 12.19/20 (60.94%, 9.38% improvement) [51] and from an average of 9.9/20 (49.5%) correct diagnoses to 14/20 (70%, 20.5% improvement) [50].…”

“…In a more recent study, Sadgehi et al [50] analyse the predictive performance of 14 non-medical graduate students and additionally evaluate whether the students perceive the system as useful for educational purposes. Both studies find improvements in the predictive performance of the graduate students when assisted by the CBIR: from on average 10.31/20 (51.56%) correct predictions to 12.19/20 (60.94%, 9.38% improvement) [51] and from an average of 9.9/20 (49.5%) correct diagnoses to 14/20 (70%, 20.5% improvement) [50]. Additionally, both studies found an increase in self-reported confidence when using the CBIR system.…”

Query Expansion Based on Top-Ranked Images for Content-Based Medical Image Retrieval

“…Most content-based medical image retrieval (CBMIR) systems are focused on image similarity, whereby a user enters a query image, and the system responds by presenting the most similar image focused on a certain similarity criterion, then, the results of the related image query are shown in descending order. The basic concept of any CBMIR method consists of two main steps or phases: extraction of the feature (offline phase) and calculation of similarity measures (online phase) [6][7][8]. Figure 1 shows the main and fundamental framework of a CBMIR system.…”