Effects of Background Colors, Flashes, and Exposure Values on the Accuracy of a Smartphone-Based Pill Recognition System Using a Deep Convolutional Neural Network: Deep Learning and Experimental Approach

Abstract

Background Pill image recognition systems are difficult to develop due to differences in pill color, which are influenced by external factors such as the illumination from and the presence of a flash. Objective In this study, the differences in color between reference images and real-world images were measured to determine the accuracy of a pill recognition system under 12 real-world conditions (ie, different background colors, the presence and absence of a flash, and different exposure values [EVs]). Methods We analyzed 19 medications with different features (ie, different colors, shapes, and dosages). The average color difference was calculated based on the color distance between a reference image and a real-world image. Results For images with black backgrounds, as the EV decreased, the top-1 and top-5 accuracies increased independently of the presence of a flash. The top-5 accuracy for images with black backgrounds increased from 26.8% to 72.6% when the flash was on and increased from 29.5% to 76.8% when the flash was off as the EV decreased. However, the top-5 accuracy increased from 62.1% to 78.4% for images with white backgrounds when the flash was on. The best top-1 accuracy was 51.1% (white background; flash on; EV of +2.0). The best top-5 accuracy was 78.4% (white background; flash on; EV of 0). Conclusions The accuracy generally increased as the color difference decreased, except for images with black backgrounds and an EV of −2.0. This study revealed that background colors, the presence of a flash, and EVs in real-world conditions are important factors that affect the performance of a pill recognition model.

A Novel Two-Stage Induced Deep Learning System for Classifying Similar Drugs with Diverse Packaging

“…Recently, some studies have used smartphones for practical deep-learning-based drug-classification systems and achieved a good performance [ 19 ]. Hence, in this study, we capture the drug images by using the main camera of the Samsung Note 20 Ultra smartphone to better match future practical applications.…”

“…Patel U. et al reviewed the relevant literature, highlighting the potential applications of drug classification in pharmaceutical manufacturing and packaging and emphasizing the potential environmental factors that could impact it [ 18 ]. Cha K. et al investigated the influence of illumination sources and backgrounds on medication classification [ 19 ]. Recently, Zheng A. et al applied the Bidirectional Generative Adversarial networks (Bi-GAN) to classify four categories of unpackaged pills by using near-infrared spectroscopy (NIS).…”

Technology Behavior Model—Beyond Your Sight with Extended Reality in Surgery

“…Past works of literature on XRT in surgery have been insufficiently discussed. However, from the point of view of theoretical development, the XRSG technique has met the needs of surgical procedures, such as trials in dermatology [14], neurosurgery [9], and urological procedures [15], but has not yet been applied to clinical procedures [16]. In this study, the acceptability and feasibility of the use of XRSG for surgical operations by medical specialists were tested.…”

“…Third, the contributions of this study are discussed, including the proposed new technical-behavioral model for medical image modeling by using Extended Reality Technology, which improves the Usage Perspicuity, standardizes the operation of medical experts while allowing the use of multiple interaction methods for viewing images and data, and avoids cross-contamination in the surgical environment. This provides a better patient-to- However, from the point of view of theoretical development, the XRSG technique has met the needs of surgical procedures, such as trials in dermatology [14], neurosurgery [9], and urological procedures [15], but has not yet been applied to clinical procedures [16]. In this study, the acceptability and feasibility of the use of XRSG for surgical operations by medical specialists were tested.…”