An Algorithm for Digital Medicine Testing: A NODE.Health Perspective Intended to Help Emerging Technology Companies and Healthcare Systems Navigate the Trial and Testing Period prior to Full-Scale Adoption

Sheon, Amy R.; Winkle, Brian Van; Solad, Yauheni; Atreja, Ashish

doi:10.1159/000494365

Cited by 3 publications

(1 citation statement)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A previous study showed that CM-SHARE, a voluntary-use digital health solution, was successfully integrated into a real-world primary care setting with high adoption and consistent use in caring for patients with cardiometabolic conditions [ 12 ]. In this study, following principles of the digital health technology development and deployment [ 23 , 24 ], we first tested CM-SHARE among a small group of pilot users (ie, PCPs) before spreading to a much broader group of PCP end users. Although there is an increasing availability of digital tools created by health care or high-technology companies, little is known about whether the impact of these digital tools on pilot users is sustained when the technology is spread to a much broader group of users [ 25 , 26 ].…”

Section: Introductionmentioning

confidence: 99%

Evaluating the Impact of a Point-of-Care Cardiometabolic Clinical Decision Support Tool on Clinical Efficiency Using Electronic Health Record Audit Log Data: Algorithm Development and Validation

Yan¹,

Husby²,

Mudiganti³

et al. 2022

JMIR Med Inform

View full text Add to dashboard Cite

Background Electronic health record (EHR) systems are becoming increasingly complicated, leading to concerns about rising physician burnout, particularly for primary care physicians (PCPs). Managing the most common cardiometabolic chronic conditions by PCPs during a limited clinical time with a patient is challenging. Objective This study aimed to evaluate a Cardiometabolic Sutter Health Advanced Reengineered Encounter (CM-SHARE), a web-based application to visualize key EHR data, on the EHR use efficiency. Methods We developed algorithms to identify key clinic workflow measures (eg, total encounter time, total physician time in the examination room, and physician EHR time in the examination room) using audit data, and we validated and calibrated the measures with time-motion data. We used a pre-post parallel design to identify propensity score–matched CM-SHARE users (cases), nonusers (controls), and nested-matched patients. Cardiometabolic encounters from matched case and control patients were used for the workflow evaluation. Outcome measures were compared between the cases and controls. We applied this approach separately to both the CM-SHARE pilot and spread phases. Results Time-motion observation was conducted on 101 primary care encounters for 9 PCPs in 3 clinics. There was little difference (<0.8 minutes) between the audit data–derived workflow measures and the time-motion observation. Two key unobservable times from audit data, physician entry into and exiting the examination room, were imputed based on time-motion studies. CM-SHARE was launched with 6 pilot PCPs in April 2016. During the prestudy period (April 1, 2015, to April 1, 2016), 870 control patients with 2845 encounters were matched with 870 case patients and encounters, and 727 case patients with 852 encounters were matched with 727 control patients and 3754 encounters in the poststudy period (June 1, 2016, to June 30, 2017). Total encounter time was slightly shorter (mean −2.7, SD 1.4 minutes, 95% CI −4.7 to −0.9; mean –1.6, SD 1.1 minutes, 95% CI −3.2 to −0.1) for cases than controls for both periods. CM-SHARE saves physicians approximately 2 minutes EHR time in the examination room (mean −2.0, SD 1.3, 95% CI −3.4 to −0.9) compared with prestudy period and poststudy period controls (mean −1.9, SD 0.9, 95% CI −3.8 to −0.5). In the spread phase, 48 CM-SHARE spread PCPs were matched with 84 control PCPs and 1272 cases with 3412 control patients, having 1119 and 4240 encounters, respectively. A significant reduction in total encounter time for the CM-SHARE group was observed for short appointments (≤20 minutes; 5.3-minute reduction on average) only. Total physician EHR time was significantly reduced for both longer and shorter appointments (17%-33% reductions). Conclusions Combining EHR audit log files and clinical information, our approach offers an innovative and scalable method and new measures that can be used to evaluate clinical EHR efficiency of digital tools used in clinical settings.

show abstract

Section: Introductionmentioning

confidence: 99%

Evaluating the Impact of a Point-of-Care Cardiometabolic Clinical Decision Support Tool on Clinical Efficiency Using Electronic Health Record Audit Log Data: Algorithm Development and Validation

Yan¹,

Husby²,

Mudiganti³

et al. 2022

JMIR Med Inform

View full text Add to dashboard Cite

show abstract

Design and testing of a mobile health application rating tool

Levine

Newmark

et al. 2020

npj Digit. Med.

View full text Add to dashboard Cite

Mobile health applications (“apps”) have rapidly proliferated, yet their ability to improve outcomes for patients remains unclear. A validated tool that addresses apps’ potentially important dimensions has not been available to patients and clinicians. The objective of this study was to develop and preliminarily assess a usable, valid, and open-source rating tool to objectively measure the risks and benefits of health apps. We accomplished this by using a Delphi process, where we constructed an app rating tool called THESIS that could promote informed app selection. We used a systematic process to select chronic disease apps with ≥4 stars and <4-stars and then rated them with THESIS to examine the tool’s interrater reliability and internal consistency. We rated 211 apps, finding they performed fair overall (3.02 out of 5 [95% CI, 2.96–3.09]), but especially poorly for privacy/security (2.21 out of 5 [95% CI, 2.11–2.32]), interoperability (1.75 [95% CI, 1.59–1.91]), and availability in multiple languages (1.43 out of 5 [95% CI, 1.30–1.56]). Ratings using THESIS had fair interrater reliability (κ = 0.3–0.6) and excellent scale reliability (ɑ = 0.85). Correlation with traditional star ratings was low (r = 0.24), suggesting THESIS captures issues beyond general user acceptance. Preliminary testing of THESIS suggests apps that serve patients with chronic disease could perform much better, particularly in privacy/security and interoperability. THESIS warrants further testing and may guide software and policymakers to further improve app performance, so apps can more consistently improve patient outcomes.

show abstract