Exploring the analysis capabilities and clinical application potential of the Claude 3 Opus in different dermatologic images: the development of a large-scale multimodal model to assist in dermatology clinical practice (Preprint)

Abstract

BACKGROUND Publicly available, accessible, and user-friendly artificial intelligence is expected to serve in medical processes. Claude3, a newly introduced large-scale multimodal model, has demonstrated significantly superior image analysis capabilities compared to other models in official tests. However, there is no research reporting the potential of Claude3 in medical image analysis. OBJECTIVE To explore the analysis capabilities and potential applications of Claude3 Opus on dermatologic images. METHODS Dermoscopy and dermatopathology images from textbooks were collected, and question templates for different types of diseases were designed for Claude3 Opus. Three dermatologists used a structured scoring system with five modules to evaluate Claude3 Opus' analysis of images based on recognition, description, completeness, diagnosis, and clinical application, with each module scored out of 5 and a total score of 25. RESULTS A total of 330 images were collected. Claude3 Opus scored highest in pigmented disorders dermoscopy images (18.65/25), followed by vascular disorders dermoscopy (15.97/25) and vascular disorders dermatopathology images (15.86/25). In pigmented disorders, its score in dermoscopy (18.65/25) was significantly higher than in dermatopathology images (14.54/25), but no such difference existed in vascular disorders. Among the five modules, Claude3 Opus' recognition score (3.65/5) was significantly higher than the other four modules. There was no difference between description score (3.14/5) and completeness score (3.22/5), but they were significantly higher than the diagnostic score (2.47/5). Claude3 Opus scored higher in malignant diseases than benign diseases, regardless of dermoscopy or dermatopathology images (all p-values <0.05), with no impact from different magnifications in dermatopathology images (p>0.05) and no impact from different number of evaluators. CONCLUSIONS Claude3 Opus exhibits strong recognition capabilities for dermatologic disease images, can accurately describe abnormalities in images completely, and shows high sensitivity to malignant diseases. Apart from medical assistance, Claude3 Opus could potentially be widely used in medical education and patient communication. CLINICALTRIAL no need