Automatic range of motion measurement via smartphone images for telemedicine examination of the hand

Abstract

Background Telemedicine support virtual consultations and evaluations in hand surgery for patients in remote areas during the COVID-19 era. However, traditional physical examination is challenging in telemedicine and it is inconvenient to manually measure the hand range of motion (ROM) from images or videos. Here, we propose an automatic method using the hand pose estimation technique, aiming to measure the hand ROM from smartphone images. Methods Twenty-eight healthy volunteers participated in the study. An eight-hand gestures measurement protocol and the Google MediaPipe Hands were used to analyze images and calculate the ROM automatically. Manual goniometry was also performed according to the guideline of the American Medical Association. The correlation between the automatic and manual methods was analyzed by the intraclass correlation coefficient and Pearson correlation coefficient. The clinical acceptance was testified using Bland-Altman plots. Results A total of 32 parameters of each hand were measured by both methods, and 1792 measurement results were compared. The mean difference between automatic and manual methods is −2.21 ± 9.29° in the angle measurement and 0.48 ± 0.48 cm in the distance measurement. The intraclass correlation coefficient of 75% of parameters was higher than 0.75, the Pearson correlation coefficient of 84% of parameters was over 0.6, and 40.6% of parameters reached well-accepted clinical agreements. Conclusions The proposed method provides a helpful protocol for automatic hand ROM measurement based on smartphone images and the MediaPipe Hands pose estimation technique. The automatic measurement is acceptable and comparable with existing methods, showing a possible application in the telemedicine examination of hand surgery.

Estimating the Thumb Rotation Angle by Using a Tablet Device With a Posture Estimation Artificial Intelligence Model

“…This level of accuracy was favorable even compared to previous research. Gu et al have reported that the average difference from the goniometer was −2.21 ± 9.29° when analyzing images of a specific protocol position captured with a smartphone using only MediaPipe [ 4 ]. Therefore, we combined MediaPipe and ML algorithms.…”

“…However, its measurement requires face-to-face evaluation and physical contact. This has become particularly challenging after the coronavirus disease 2019 (COVID-19) pandemic, thereby highlighting the importance of telemedicine for remote consultations and evaluations [ 4 ].…”

“…Although reports of the range of motion assessment for hands using MediaPipe have emerged, the evaluation of automated hand motion alone using MediaPipe with smartphone camera functionality lacks accuracy. When analyzing images of a specific protocol position captured with a smartphone, the average difference from the goniometer was −2.21 ± 9.29°[ 4 ].…”

“…Measuring the range of motion of the hand enables hand surgeons and therapists to assess hand function objectively and quantitatively [1][2][3][4]. An accurate measurement of the kinematics of the thumb can be achieved by using several well-known techniques, and optical systems featuring infrared cameras and reflective markers are considered the gold standard method [5].…”

Tap and Swipe Smartphone Gestures Indicating Hand Tremor and Finger Joint Range in Motion

“…This method is not easily operated [9] and requires the hands to be rigidly held in static positions. Using computer vision algorithms, images of the hands are analyzed for joint angles, with comparable results to the goniometer [2,15,16]. However, this requires mounts to position a smartphone at the required distance and instructions for keeping the hand in the camera's view frame.…”