Patient's Perception of Digital Symptom Assessment Technologies in Rheumatology: Results From a Multicentre Study

Knitza, Johannes; Muehlensiepen, Felix; Ignatyev, Yuriy; Fuchs, Franziska; Mohn, Jacob; Simón, David; Kleyer, Arnd; Fagni, Filippo; Boeltz, Sebastian; Morf, Harriet; Bergmann, Christina; Labinsky, Hannah; Vorbrüggen, Wolfgang; Ramming, Andreas; Distler, Jörg H W; Bartz‐Bazzanella, Peter; Vuillerme, Nicolas; Schett, Georg; Welcker, Martin; Hueber, Axel J.

doi:10.3389/fpubh.2022.844669

Cited by 26 publications

(18 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Real-world evidence studies involving real patients have been recommended to examine the performance of symptom checkers against clinicians 27 . The importance of using real-world data within digital health technology development is also strongly supported by regulatory authorities, as evidenced by the introduction of the ‘Real World Evidence’ program by the FDA, which have also approved databases with real-world patient data to support this initiative 31 – 33 .…”

Section: Discussionmentioning

confidence: 99%

The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review

et al. 2022

View full text Add to dashboard Cite

Digital and online symptom checkers are an increasingly adopted class of health technologies that enable patients to input their symptoms and biodata to produce a set of likely diagnoses and associated triage advice. However, concerns regarding the accuracy and safety of these symptom checkers have been raised. This systematic review evaluates the accuracy of symptom checkers in providing diagnoses and appropriate triage advice. MEDLINE and Web of Science were searched for studies that used either real or simulated patients to evaluate online or digital symptom checkers. The primary outcomes were the diagnostic and triage accuracy of the symptom checkers. The QUADAS-2 tool was used to assess study quality. Of the 177 studies retrieved, 10 studies met the inclusion criteria. Researchers evaluated the accuracy of symptom checkers using a variety of medical conditions, including ophthalmological conditions, inflammatory arthritides and HIV. A total of 50% of the studies recruited real patients, while the remainder used simulated cases. The diagnostic accuracy of the primary diagnosis was low across included studies (range: 19–37.9%) and varied between individual symptom checkers, despite consistent symptom data input. Triage accuracy (range: 48.8–90.1%) was typically higher than diagnostic accuracy. Overall, the diagnostic and triage accuracy of symptom checkers are variable and of low accuracy. Given the increasing push towards adopting this class of technologies across numerous health systems, this study demonstrates that reliance upon symptom checkers could pose significant patient safety hazards. Large-scale primary studies, based upon real-world data, are warranted to demonstrate the adequate performance of these technologies in a manner that is non-inferior to current best practices. Moreover, an urgent assessment of how these systems are regulated and implemented is required.

show abstract

Section: Discussionmentioning

confidence: 99%

The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…A study in California [ 18 ] also examined the mix of patients using Ada in a health system and found that the patient characteristics of users were similar to those of their general patient population but with a younger mean age. A study by Knitza et al [ 38 ] on the use of Ada, a custom diagnostic app (Rheport), and web-based searching for symptoms, showed a high System Usability Scale score of 77.1 for Rheport and a somewhat lower score for Ada of 74.4 ( P <.001). Ratings for “very helpful” or “helpful” were higher for Rheport (65.8% vs 44.3%), although similar numbers would recommend each system to others (79.2% vs 73.3%).…”

Section: Discussionmentioning

confidence: 99%

Evaluation of Diagnostic and Triage Accuracy and Usability of a Symptom Checker in an Emergency Department: Observational Study

Fraser¹,

Cohan²,

Koehler³

et al. 2022

JMIR Mhealth Uhealth

View full text Add to dashboard Cite

Background Symptom checkers are clinical decision support apps for patients, used by tens of millions of people annually. They are designed to provide diagnostic and triage advice and assist users in seeking the appropriate level of care. Little evidence is available regarding their diagnostic and triage accuracy with direct use by patients for urgent conditions. Objective The aim of this study is to determine the diagnostic and triage accuracy and usability of a symptom checker in use by patients presenting to an emergency department (ED). Methods We recruited a convenience sample of English-speaking patients presenting for care in an urban ED. Each consenting patient used a leading symptom checker from Ada Health before the ED evaluation. Diagnostic accuracy was evaluated by comparing the symptom checker’s diagnoses and those of 3 independent emergency physicians viewing the patient-entered symptom data, with the final diagnoses from the ED evaluation. The Ada diagnoses and triage were also critiqued by the independent physicians. The patients completed a usability survey based on the Technology Acceptance Model. Results A total of 40 (80%) of the 50 participants approached completed the symptom checker assessment and usability survey. Their mean age was 39.3 (SD 15.9; range 18-76) years, and they were 65% (26/40) female, 68% (27/40) White, 48% (19/40) Hispanic or Latino, and 13% (5/40) Black or African American. Some cases had missing data or a lack of a clear ED diagnosis; 75% (30/40) were included in the analysis of diagnosis, and 93% (37/40) for triage. The sensitivity for at least one of the final ED diagnoses by Ada (based on its top 5 diagnoses) was 70% (95% CI 54%-86%), close to the mean sensitivity for the 3 physicians (on their top 3 diagnoses) of 68.9%. The physicians rated the Ada triage decisions as 62% (23/37) fully agree and 24% (9/37) safe but too cautious. It was rated as unsafe and too risky in 22% (8/37) of cases by at least one physician, in 14% (5/37) of cases by at least two physicians, and in 5% (2/37) of cases by all 3 physicians. Usability was rated highly; participants agreed or strongly agreed with the 7 Technology Acceptance Model usability questions with a mean score of 84.6%, although “satisfaction” and “enjoyment” were rated low. Conclusions This study provides preliminary evidence that a symptom checker can provide acceptable usability and diagnostic accuracy for patients with various urgent conditions. A total of 14% (5/37) of symptom checker triage recommendations were deemed unsafe and too risky by at least two physicians based on the symptoms recorded, similar to the results of studies on telephone and nurse triage. Larger studies are needed of diagnosis and triage performance with direct patient use in different clinical environments.

show abstract

“…44 Nearly half of all patients used an online search engine prior to their first rheumatology appointment. 12 As current symptom checkers are not very reliable, 14 self-sampling kits could be sent to patients to improve diagnostic accuracy in a second step. These results and any additional diagnostic procedures could be discussed virtually via RMD Open RMD Open RMD Open video consultations.…”

Section: Discussionmentioning

confidence: 99%

Remote self-collection of capillary blood using upper arm devices for autoantibody analysis in patients with immune-mediated inflammatory rheumatic diseases

et al. 2022

Self Cite

View full text Add to dashboard Cite

ObjectivesTo evaluate the feasibility, accuracy, usability and acceptability of two upper arm self-sampling devices for measurement of autoantibodies and C reactive protein (CRP) levels in patients with immune-mediated rheumatic diseases (IMRDs).Methods70 consecutive patients with IMRD with previously documented autoantibodies were assigned to supervised and unsupervised self-collection of capillary blood with the Tasso+ or TAP II device. Interchangeability of 17 biomarkers with standard venesection was assessed by: concordance, correlation, paired sample hypothesis testing and Bland-Altman plots. Patients completed an evaluation questionnaire, including the System Usability Scale (SUS) and Net Promoter Score (NPS).ResultsWhile 80.0% and 77.0% were able to safely and successfully collect capillary blood using the Tasso+ and TAP II within the first attempt, 69 of 70 (98.6%) patients were successful in collecting capillary blood within two attempts. Concordance between venous and capillary samples was high; 94.7% and 99.5% for positive and negative samples, respectively. For connective tissue disease screen, anti-Ro52 and anti-proteinase 3 autoantibody levels, no significant differences were observed. Self-sampling was less painful than standard venesection for the majority of patients (Tasso+: 71%; TAP II: 63%). Both devices were well accepted (NPS; both: +28%), usability was perceived as excellent (SUS; Tasso+: 88.6 of 100; TAP II: 86.0 of 100) and 48.6 %/62.9% of patients would prefer to use the Tasso+/TAP II, respectively, instead of a traditional venous blood collection.ConclusionsRemote self-collection of capillary blood using upper arm-based devices for autoantibody and CRP analysis in patients with autoimmune rheumatic diseases is feasible, accurate and well accepted among patients.Trial registration numberWHO International Clinical Trials Registry (DRKS00024925).

show abstract

Patient's Perception of Digital Symptom Assessment Technologies in Rheumatology: Results From a Multicentre Study

Cited by 26 publications

References 25 publications

The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review

The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review

Evaluation of Diagnostic and Triage Accuracy and Usability of a Symptom Checker in an Emergency Department: Observational Study

Remote self-collection of capillary blood using upper arm devices for autoantibody analysis in patients with immune-mediated inflammatory rheumatic diseases

Contact Info

Product

Resources

About