Convolution Neural Network Based Multi-Label Disease Detection Using Smartphone Captured Tongue Images

Abstract

Purpose: Tongue image analysis for disease diagnosis is an ancient, traditional, non-invasive diagnostic technique widely used by traditional medicine practitioners. Deep learning-based multi-label disease detection models have tremendous potential for clinical decision support systems because they facilitate preliminary diagnosis. Methods: In this work, we propose a multi-label disease detection pipeline where observation and analysis of tongue images captured and received via smartphones assist in predicting the health status of an individual. Subjects, who consult collaborating physicians, voluntarily provide all images. Images thus acquired are first and foremost classified either into a diseased or a normal category by a 5-fold cross-validation algorithm using a convolutional neural network (MobileNetV2) model for binary classification. Once it predicts the diseased label, the disease prediction algorithm based on DenseNet-121 uses the image to diagnose single or multiple disease labels. Results: The MobileNetV2 architecture-based disease detection model achieved an average accuracy of 93% in distinguishing between diseased and normal, healthy tongues, whereas the multilabel disease classification model produced more than 90% accurate results for the disease class labels considered, strongly indicating a successful outcome with the smartphone-captured image dataset. Conclusion: AI-based image analysis shows promising results, and an extensive dataset could provide further improvements to this approach. Experimenting with smartphone images opens a great opportunity to provide preliminary health status to individuals at remote locations as well, prior to further treatment and diagnosis, using the concept of telemedicine.