Color vision deficiency (CVD) affects a significant portion of the population, yet its impact is often overlooked in medical education, especially in visually demanding specialties like dermatology, pathology, and radiology. In this study, we investigated the potential of ChatGPT to comprehend CVD-simulated images in image-based diagnostic tasks. Notably, the model successfully adapted its diagnostic reasoning to match CVD-modified color perception while preserving high prediction accuracy. These findings highlight the potential of using ChatGPT to foster more inclusive learning environments for individuals with CVD in visually intensive medical specialties.