Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial

Abstract

Background Timely diagnosis and treatment are essential in the effective management of inflammatory rheumatic diseases (IRDs). Symptom checkers (SCs) promise to accelerate diagnosis, reduce misdiagnoses, and guide patients more effectively through the health care system. Although SCs are increasingly used, there exists little supporting evidence. Objective To assess the diagnostic accuracy, patient-perceived usability, and acceptance of two SCs: (1) Ada and (2) Rheport. Methods Patients newly presenting to a German secondary rheumatology outpatient clinic were randomly assigned in a 1:1 ratio to complete Ada or Rheport and consecutively the respective other SCs in a prospective non-blinded controlled randomized crossover trial. The primary outcome was the accuracy of the SCs regarding the diagnosis of an IRD compared to the physicians’ diagnosis as the gold standard. The secondary outcomes were patient-perceived usability, acceptance, and time to complete the SC. Results In this interim analysis, the first 164 patients who completed the study were analyzed. 32.9% (54/164) of the study subjects were diagnosed with an IRD. Rheport showed a sensitivity of 53.7% and a specificity of 51.8% for IRDs. Ada’s top 1 (D1) and top 5 disease suggestions (D5) showed a sensitivity of 42.6% and 53.7% and a specificity of 63.6% and 54.5% concerning IRDs, respectively. The correct diagnosis of the IRD patients was within the Ada D1 and D5 suggestions in 16.7% (9/54) and 25.9% (14/54), respectively. The median System Usability Scale (SUS) score of Ada and Rheport was 75.0/100 and 77.5/100, respectively. The median completion time for both Ada and Rheport was 7.0 and 8.5 min, respectively. Sixty-four percent and 67.1% would recommend using Ada and Rheport to friends and other patients, respectively. Conclusions While SCs are well accepted among patients, their diagnostic accuracy is limited to date. Trial registration DRKS.de, DRKS00017642. Registered on 23 July 2019

Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students

“…A first rheumatology-specific study with 34 patients using symptom checkers showed that only 4 out of 21 patients with inflammatory arthritis were given the first diagnosis of rheumatoid arthritis or psoriatic arthritis [ 31 ]. Importantly, we could recently show that the same DDSS, when used by patients with inflammatory rheumatic diseases, displayed the correct diagnoses among the first and overall disease suggestions in 17% (9/54) and 26% (14/54) of the time [ 32 ]. Surprisingly, we found no correlation between the proportion of symptoms correctly extracted and the DDSS diagnostic accuracy.…”

Digital Approaches for a Reliable Early Diagnosis of Psoriatic Arthritis

“…The use of mobile devices for health purposes has thus become an increasingly accessible commodity for many, and numerous health-related smartphone applications are now available ( 64 , 65 ). These include symptom checkers ( 66 , 67 ) and referral tools ( 68 ) that could be helpful for the screening of early arthritic manifestations in PsO, as well as applications for the passive and active monitoring of disease activity [e.g., wearable sensors applications for the passive monitoring of key disease features ( 45 , 69 , 70 ), self-reporting tools ( 71 )]. Medication adherence could also be improved via notification reminders on personal smartphones ( 72 – 74 ).…”

“…Symptom checkers are artificial-intelligence (AI)-based tools designed to help patients determining the possible causes of their symptoms and eventually direct them to medical attention. In recent years, the use of internet- and mobile- based support for self-diagnosing has greatly increased, and an increasing number of patients perceive symptom checkers as useful for diagnosis ( 67 , 91 ). In rheumatology, symptom checkers could help reducing diagnostic delay by ensuring quicker referral of patients who are at higher risk of having or developing a rheumatic condition.…”

Digitalisierung in der Rheumapraxis

“…In Deutschland sind aus Sicht der Rheumatologie vor allen Dingen die Applikation ADA ( www.ada.de ) und RhePORT ( www.rheport.de ) [ 5 ] zu nennen. In einer multizentrischen Studie erreichten beide Systeme eine vergleichbare Diagnosegenauigkeit [ 6 ]. ADA gibt dem Nutzer nach Durchlaufen der Fragen des Algorithmus 4 bis 5 gewichtete mögliche Diagnosen und schlägt vor, diese beim Facharzt abklären zu lassen.…”

“…rheport.de) [5] zu nennen. In einer multizentrischen Studie erreichten beide Systeme eine vergleichbare Diagnosegenauigkeit [6]. ADA gibt dem Nutzer nach Durchlaufen der Fragen des Algorithmus Auch bereits bekannte Patienten müssen sich regelmäßig an der Anmeldung bzw.…”