Computer vision detects inflammatory arthritis in standardized smartphone photographs in an Indian patient cohort

Abstract

IntroductionComputer vision extracts meaning from pixelated images and holds promise in automating clinical tasks. Convolutional neural networks (CNN), deep learning networks used therein, have shown promise in X-ray images as well as joint photographs. We studied the performance of a CNN on standardized smartphone photographs in detecting inflammation in three hand joints.MethodsWe enrolled consecutive patients with inflammatory arthritis of less than two years duration and excluded those with deformities. Each patient was examined by a rheumatologist and the presence of synovitis in each joint was recorded. Hand photographs were taken in a standardized manner and anonymized. Images were cropped to include joints of interest. A reNrt-101 backbone modified for two class outputs (inflamed or not) was used for training. We also tested a hue augmented dataset. We report accuracy, sensitivity and specificity for three joints: wrist, index finger proximal interphalangeal (IFPIP), middle finger interphalangeal (MFPIP).ResultsThe cohort had a mean age of 49.7 years; most had rheumatoid arthritis(n=68). The wrist (62.5%), MFPIP (47%) and IFPIP (41.5%) were the three most commonly inflamed joints. The CNN achieved the highest accuracy in being able to detect synovitis in the MFPIP (83%) followed by the IFPIP (74%) and the wrist (65%).DiscussionWe show that computer vision was able to detect inflammation in three joints of the hand with reasonable accuracy on standardized photographs despite a small dataset. Feature engineering was not required, and the CNN worked despite a diversity in clinical diagnosis. Larger datasets are likely to improve accuracy and help explain the basis of classification. These data suggest a potential use of computer vision in screening and follow-up of inflammatory arthritis.