Quantifying Patient Portal Use: Systematic Review of Utilization Metrics (Preprint)

Abstract

BACKGROUND Use of patient portals has been associated with positive outcomes in patient engagement and satisfaction; portal studies have also connected portal use as well as the nature of users’ interactions with portals and the contents of their generated data to meaningful cost and quality outcomes. Incentive programs in the USA have encouraged uptake of health information technology, including portals, by setting standards for meaningful use of such technology. However, despite widespread interest in patient portal use and adoption, studies on patient portals differ in actual metrics used to operationalize and track utilization, leading to unsystematic and incommensurable characterizations of use. OBJECTIVE No review has systematically assessed the measurements used to investigate patient portal utilization. Therefore, the objective of this study is to apply systematic review criteria to identify and compare methods for quantifying and reporting patient portal use. METHODS Original studies with quantifiable metrics of portal use, available in English between 2014 and the search date of October 17, 2018, were obtained from PubMed, using the Medical Subject Heading term “patient portals” and related keyword searches. The first search round included full text review of all results to confirm a priori data charting elements of interest and suggest additional categories inductively; this round was supplemented by retrieval of works cited in systematic reviews (based on title screening). An additional search round included broader keywords. Studies were screened at abstract-level for inclusion, confirmed by at least two raters. Included studies were analyzed for metrics related to basic use / adoption; frequency of use; duration metrics; intensity; and stratification of users into “super user” or high-utilization types. Additional categories related to provider (including care team / administrative) use of the portal were identified inductively. Analysis of whether metrics aligned with Meaningful Use Stage 2 categories (MU-2) employed by the US Centers for Medicare and Medicaid Services was also conducted. RESULTS Of 315 distinct search results, 87 met inclusion criteria. Of the a priori metrics, plus provider use, most studies included either three types (27 studies, 31.03%) or four types (22, 25.29%) of metrics. Nine studies (10.34%) only reported the patient use / adoption metric, and only one study (1.15%) reported all six. Of the U.S.-based studies (76), 18 were explicitly motivated by MU-2 compliance; 40 (52.63%) at least mentioned these incentives, but only 6 (7.89%) presented metrics from which compliance rates could be inferred. CONCLUSIONS Portal utilization measures in the research literature can fall below established standards for “meaningful” or they can substantively exceed those standards in the type and number of utilization properties measured. Understanding how patient portal use has been defined and operationalized may encourage more consistent, well-defined, and perhaps more meaningful standards for utilization, informing future portal development.

Investigating Popular Mental Health Mobile Application Downloads and Activity During the COVID-19 Pandemic

“…Variables such as frequency of usage, duration of each use, and length of engagement have been studied as viable metrics to identify engaged users (Chen et al, 2019); however, currently the detailed usage data of an app is only available when the developers choose to track and share such data. Similar issues were found in a study on patient portals (Beal et al, 2020).…”