Usability Testing of a Digital Assessment Routing Tool for Musculoskeletal Disorders: Iterative, Convergent Mixed Methods Study

Lowe, Cabella; Browne, Mitchell; Marsh, William; Morrissey, Dylan

doi:10.2196/38352

Cited by 3 publications

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Investigating the Use of Telemedicine by Health Care Providers to Diagnose and Manage Patients With Musculoskeletal Disorders: Systematic Review and Meta-Analysis

Vincent,

Charron,

Lafrance

et al. 2024

J Med Internet Res

View full text Add to dashboard Cite

Background Access to care is a major challenge for patients with musculoskeletal disorders (MSKDs). Telemedicine is one of the solutions to improve access to care. However, initial remote diagnosis of MSKDs involves some challenges, such as the impossibility of touching the patient during the physical examination, which makes it more complex to obtain a valid diagnosis. No meta-analysis has been performed to date to synthesize evidence regarding the initial assessment including a physical evaluation using telemedicine to diagnose patients with MSKDs. Objective This study aims to appraise the evidence on diagnostic and treatment plan concordance between remote assessment using synchronous or asynchronous forms of telemedicine and usual in-person assessment for the initial evaluation of various MSKDs. Methods An electronic search was conducted up to August 2023 using terms related to telemedicine and assessment of MSKDs. Methodological quality of studies was assessed with the Quality Assessment of Diagnostic Accuracy Studies 2 tool. Random-effect model meta-analyses were performed. The Grading of Recommendations, Assessment, Development, and Evaluations framework was used to synthesize the quality and certainty of the evidence. Results A total of 23 concordance studies were eligible and included adult participants (N=1493) with various MSKDs. On the basis of high certainty, pooled κ and prevalence-adjusted and bias-adjusted κ for the diagnostic concordance between remote and in-person assessments of MSKDs were 0.80 (95% CI 0.72-0.89; 7 studies, 353 patients) and 0.83 (95% CI 0.76-0.89; 6 studies, 306 patients). On the basis of moderate certainty, pooled Gwet AC1 for treatment plan concordance between remote and in-person assessments of MSKDs was 0.90 (95% CI 0.80-0.99; 2 studies, 142 patients). Conclusions The diagnostic concordance for MSKDs is good to very good. Treatment plan concordance is probably good to excellent. Studies evaluating the accuracy to detect red and yellow flags as well as the potential increase in associated health care resources use, such as imaging tests, are needed.

show abstract

Investigating the Use of Telemedicine by Health Care Providers to Diagnose and Manage Patients With Musculoskeletal Disorders: Systematic Review and Meta-Analysis

Vincent,

Charron,

Lafrance

et al. 2024

J Med Internet Res

View full text Add to dashboard Cite

show abstract

Evaluation of a Musculoskeletal Digital Assessment Routing Tool (DART): Crossover Noninferiority Randomized Pilot Trial

Lowe,

Sephton,

Marsh

et al. 2024

JMIR Form Res

View full text Add to dashboard Cite

Background Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on patients and increasing demand for health care use. Triage directing patients to appropriate level intervention improving health outcomes and efficiency has been prioritized. We developed a musculoskeletal digital assessment routing tool (DART) mobile health (mHealth) system, which requires evaluation prior to implementation. Such innovations are rarely rigorously tested in clinical trials—considered the gold standard for evaluating safety and efficacy. This pilot study is a precursor to a trial assessing DART performance with a physiotherapist-led triage assessment. Objective The study aims to evaluate trial design, assess procedures, and collect exploratory data to establish the feasibility of delivering an adequately powered, definitive randomized trial, assessing DART safety and efficacy in an NHS primary care setting. Methods A crossover, noninferiority pilot trial using an integrated knowledge translation approach within a National Health Service England primary care setting. Participants were patients seeking assessment for a musculoskeletal condition, completing a DART assessment and the history-taking element of a face-to-face physiotherapist-led triage in a randomized order. The primary outcome was agreement between DART and physiotherapist triage recommendation. Data allowed analysis of participant recruitment and retention, randomization, blinding, study burden, and potential barriers to intervention delivery. Participant satisfaction was measured using the System Usability Scale. Results Over 8 weeks, 129 patients were invited to participate. Of these, 92% (119/129) proceeded to eligibility assessment, with 60% (78/129) meeting the inclusion criteria and being randomized into each intervention arm (39/39). There were no dropouts and data were analyzed for all 78 participants. Agreement between physiotherapist and DART across all participants and all primary triage outcomes was 41% (32/78; 95% CI 22-45), intraclass correlation coefficient 0.37 (95% CI 0.16-0.55), indicating that the reliability of DART was poor to moderate. Feedback from the clinical service team led to an adjusted analysis yielding of 78% (61/78; 95% CI 47-78) and an intraclass correlation coefficient of 0.57 (95% CI 0.40-0.70). Participant satisfaction was measured quantitively using amalgamated System Usability Scale scores (n=78; mean score 84.0; 90% CI +2.94 to –2.94), equating to an “excellent” system. There were no study incidents, and the trial burden was acceptable. Conclusions Physiotherapist-DART agreement of 78%, with no adverse triage decisions and high patient satisfaction, was sufficient to conclude DART had the potential to improve the musculoskeletal pathway. Study validity was enhanced by the recruitment of real-world patients and using an integrated knowledge translation approach. Completion of a context-specific consensus process is recommended to provide definitive definitions of safety criteria, range of appropriateness, noninferiority margin, and sample size. This pilot demonstrated an adequately powered definitive trial is feasible, which would provide evidence of DART safety and efficacy, ultimately informing potential for DART implementation. Trial Registration ClinicalTrials.gov NCT04904029; http://clinicaltrials.gov/ct2/show/NCT04904029 International Registered Report Identifier (IRRID) RR2-10.2196/31541

show abstract

Improving safety, efficiency, cost, and satisfaction across a musculoskeletal pathway using the Digital Assessment Routing tool for triage: a quality improvement study. (Preprint)

Lowe,

Atherton,

Lloyd

et al. 2024

Preprint

View full text Add to dashboard Cite

BACKGROUND The increasing prevalence of musculoskeletal conditions results in an increasing financial burden to societies and healthcare systems. Triaging patients safely and effectively improves outcomes and reduces costs across the pathway, with digital solutions offering potential advantages over traditional methods. OBJECTIVE We aimed to examine the impact of introducing DART (Digital Assessment Routing Tool) on safety, efficiency, cost and satisfaction across a National Health Service England musculoskeletal service. METHODS A quality improvement project utilizing a Plan-Do-Study-Act design and Integrated Knowledge Translation model. All musculoskeletal self-referring patients completed an online DART assessment independently, or with administrative telephone support. Quantitative and qualitative methods evaluated the primary outcome of safety, indicated both by agreement between clinician judgement and DART stratification, and safety incident surveillance. Secondary outcomes included efficiency, cost and satisfaction. RESULTS Over 4 months 4076 patients self-referred, 93% self-assessing using DART and the remainder via an administrator. Agreement between clinicians and DART was 96%, no safety incidents occurred, and 203 fewer cases required clinical escalation at initial clinician contact compared to the pre-project period. Administrative time to process self-referrals reduced by 83% with a projected £15,312 (US$20,067)/annum cost-saving. Routing to self-management and an osteoarthritis knee program would reduce costs by £79,590 (US$104,310)/annum when implemented. Further potential savings of £28,476 (US$37,320)/annum were demonstrated by DART screening for service eligibility criteria. Patient satisfaction was 90%, with all 14 administrators and 40 clinicians rating the introduction of DART as a positive service improvement. CONCLUSIONS The introduction of DART demonstrated positive outcomes in all measures presenting opportunities to improve safety and efficiency, reduce cost and improve patient and clinician satisfaction. In addition, the successful delivery of an Integrated Knowledge Translation Approach provides a model for other researchers wishing to test and implement digital health within a musculoskeletal service.

show abstract

Usability Testing of a Digital Assessment Routing Tool for Musculoskeletal Disorders: Iterative, Convergent Mixed Methods Study

Cited by 3 publications

References 59 publications

Investigating the Use of Telemedicine by Health Care Providers to Diagnose and Manage Patients With Musculoskeletal Disorders: Systematic Review and Meta-Analysis

Investigating the Use of Telemedicine by Health Care Providers to Diagnose and Manage Patients With Musculoskeletal Disorders: Systematic Review and Meta-Analysis

Evaluation of a Musculoskeletal Digital Assessment Routing Tool (DART): Crossover Noninferiority Randomized Pilot Trial

Improving safety, efficiency, cost, and satisfaction across a musculoskeletal pathway using the Digital Assessment Routing tool for triage: a quality improvement study. (Preprint)

Contact Info

Product

Resources

About