Metrics for Outpatient Portal Use Based on Log File Analysis: Algorithm Development

Tosto, Gennaro Di; McAlearney, Ann Scheck; Fareed, Naleef; Huerta, Timothy R.

doi:10.2196/16849

Cited by 23 publications

(26 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, the published literature is limited to analysis that is mostly based on patient portal utilization, as defined by MU criteria. More in-depth studies, mirroring the log file analysis conducted by Huerta et al [ 105 ] and, more recently, Di Tosto et al [ 115 ] that included a blueprint of individual patients’ portal actions, would fulfill our endeavor to utilize patient portal data more completely than the literature currently reports routinely. A systematic approach to measurement of portal usage is necessary to more readily draw comparisons across existing and future studies.…”

Section: Discussionmentioning

confidence: 95%

Quantifying Patient Portal Use: Systematic Review of Utilization Metrics

Beal¹,

Kolman²,

Jones³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

Background Use of patient portals has been associated with positive outcomes in patient engagement and satisfaction. Portal studies have also connected portal use, as well as the nature of users’ interactions with portals, and the contents of their generated data to meaningful cost and quality outcomes. Incentive programs in the United States have encouraged uptake of health information technology, including patient portals, by setting standards for meaningful use of such technology. However, despite widespread interest in patient portal use and adoption, studies on patient portals differ in actual metrics used to operationalize and track utilization, leading to unsystematic and incommensurable characterizations of use. No known review has systematically assessed the measurements used to investigate patient portal utilization. Objective The objective of this study was to apply systematic review criteria to identify and compare methods for quantifying and reporting patient portal use. Methods Original studies with quantifiable metrics of portal use published in English between 2014 and the search date of October 17, 2018, were obtained from PubMed using the Medical Subject Heading term “Patient Portals” and related keyword searches. The first search round included full text review of all results to confirm a priori data charting elements of interest and suggest additional categories inductively; this round was supplemented by the retrieval of works cited in systematic reviews (based on title screening of all citations). An additional search round included broader keywords identified during the full-text review of the first round. Second round results were screened at abstract level for inclusion and confirmed by at least two raters. Included studies were analyzed for metrics related to basic use/adoption, frequency of use, duration metrics, intensity of use, and stratification of users into “super user” or high utilizers. Additional categories related to provider (including care team/administrative) use of the portal were identified inductively. Additional analyses included metrics aligned with meaningful use stage 2 (MU-2) categories employed by the US Centers for Medicare and Medicaid Services and the association between the number of portal metrics examined and the number of citations and the journal impact factor. Results Of 315 distinct search results, 87 met the inclusion criteria. Of the a priori metrics, plus provider use, most studies included either three (26 studies, 30%) or four (23 studies, 26%) metrics. Nine studies (10%) only reported the patient use/adoption metric and only one study (1%) reported all six metrics. Of the US-based studies (n=76), 18 (24%) were explicitly motivated by MU-2 compliance; 40 studies (53%) at least mentioned these incentives, but only 6 studies (8%) presented metrics from which compliance rates could be inferred. Finally, the number of metrics examined was not associated with either the number of citations or the publishing journal’s impact factor. Conclusions Portal utilization measures in the research literature can fall below established standards for “meaningful” or they can substantively exceed those standards in the type and number of utilization properties measured. Understanding how patient portal use has been defined and operationalized may encourage more consistent, well-defined, and perhaps more meaningful standards for utilization, informing future portal development.

show abstract

Section: Discussionmentioning

confidence: 95%

Quantifying Patient Portal Use: Systematic Review of Utilization Metrics

Beal¹,

Kolman²,

Jones³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…For instance, portal use was associated with improved glycemic control [7]. Studies have assessed outpatient portal (OPP) use by examining number of logins [8] and engagement metrics generated from log file analysis [9][10][11][12].…”

Section: Introductionmentioning

confidence: 99%

Outpatient Portal Use in Prenatal Care: Differential Use by Race, Risk, and Area Social Determinants of Health Factors

Singh¹,

Jonnalagadda²,

Morgan

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Objective: To report the relationship of Outpatient Patient Portal (OPP) use with clinical risk, area social determinants of health (SDoH), and race/ethnicity among pregnant women. Methods: Regression models predicting overall and individual portal feature use (main effects and interactions) based on key variables were specified using logfiles and clinical data. Results: Overall OPP use among non-Hispanic Black women or patients who lived in lower SDoH neighborhoods were significantly less. High-risk pregnancy patients were likely to use the OPP more than those with normal-risk pregnancy. We found similar associations with individual OPP features, like the Visit (scheduling) and My Record (test results). We also found significant interactive associations between race, clinical risk, and SDoH. Non-Hispanic Black women living in lower SDoH areas used OPP less than non-Hispanic White women from similar or affluent areas. Conclusion: More research must be conducted to learn of OPP implications for pregnant women with specific clinical diagnoses.

show abstract

“…Our estimates for SEs were clustered by patient. We did not define comprehensiveness and composite use at the sessions level, as the recording of function use may be influenced by system idiosyncrasies, such as automatically logging the patient out after 10 minutes of inactivity [19][20][21]. Second, we restricted the analysis to MCB function use to the first admission or the enrollment admission.…”

Section: Discussionmentioning

confidence: 99%

Differential Effects of Outpatient Portal User Status on Inpatient Portal Use: Observational Study (Preprint)

Fareed¹,

Jonnalagadda²,

MacEwan³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

BACKGROUND The decision to use patient portals can be influenced by multiple factors, including individuals’ perceptions of the tool, which are based on both their personal skills and experiences. Prior experience with one type of portal may make individuals more comfortable with using newer portal technologies. Experienced outpatient portal users in particular may have confidence in their ability to use inpatient portals that have similar functionality. In practice, the use of both outpatient and inpatient portal technologies can provide patients with continuity of access to their health information across care settings, but the influence of one type of portal use on the use of other portals has not been studied. OBJECTIVE This study aims to understand how patients’ use of an inpatient portal is influenced by outpatient portal use. METHODS This study included patients from an academic medical center who were provided access to an inpatient portal during their hospital stays between 2016 and 2018 (N=1571). We analyzed inpatient portal log files to investigate how inpatient portal use varied by using 3 categories of outpatient portal users: prior users, new users, and nonusers. RESULTS Compared with prior users (695/1571, 44.24%) of an outpatient portal, new users (214/1571, 13.62%) had higher use of a select set of inpatient portal functions (messaging function: incidence rate ratio [IRR] 1.33, 95% CI 1.06-1.67; function that provides access to the outpatient portal through the inpatient portal: IRR 1.34, 95% CI 1.13-1.58). Nonusers (662/1571, 42.14%), compared with prior users, had lower overall inpatient portal use (all active functions: IRR 0.68, 95% CI 0.60-0.78) and lower use of specific functions, which included the function to review vitals and laboratory results (IRR 0.51, 95% CI 0.36-0.73) and the function to access the outpatient portal (IRR 0.53, 95% CI 0.45-0.62). In comparison with prior users, nonusers also had lower odds of being comprehensive users (defined as using 8 or more unique portal functions; odds ratio [OR] 0.57, 95% CI 0.45-0.73) or composite users (defined as comprehensive users who initiated a 75th or greater percentile of portal sessions) of the inpatient portal (OR 0.42, 95% CI 0.29-0.60). CONCLUSIONS Patients’ use of an inpatient portal during their hospital stay appeared to be influenced by a combination of factors, including prior outpatient portal use. For new users, hospitalization itself, a major event that can motivate behavioral changes, may have influenced portal use. In contrast, nonusers might have lower self-efficacy in their ability to use technology to manage their health, contributing to their lower portal use. Understanding the relationship between the use of outpatient and inpatient portals can help direct targeted implementation strategies that encourage individuals to use these tools to better manage their health across care settings.

show abstract

Metrics for Outpatient Portal Use Based on Log File Analysis: Algorithm Development

Cited by 23 publications

References 17 publications

Quantifying Patient Portal Use: Systematic Review of Utilization Metrics

Quantifying Patient Portal Use: Systematic Review of Utilization Metrics

Outpatient Portal Use in Prenatal Care: Differential Use by Race, Risk, and Area Social Determinants of Health Factors

Differential Effects of Outpatient Portal User Status on Inpatient Portal Use: Observational Study (Preprint)

Contact Info

Product

Resources

About