Analytical Methods for a Learning Health System: 1. Framing the Research Question

Abstract

Learning health systems use routinely collected electronic health data (EHD) to advance knowledge and support continuous learning. Even without randomization, observational studies can play a central role as the nation’s health care system embraces comparative effectiveness research and patient-centered outcomes research. However, neither the breadth, timeliness, volume of the available information, nor sophisticated analytics, allow analysts to confidently infer causal relationships from observational data. However, depending on the research question, careful study design and appropriate analytical methods can improve the utility of EHD.The introduction to a series of four papers, this review begins with a discussion of the kind of research questions that EHD can help address, noting how different evidence and assumptions are needed for each. We argue that when the question involves describing the current (and likely future) state of affairs, causal inference is not relevant, so randomized clinical trials (RCTs) are not necessary. When the question is whether an intervention improves outcomes of interest, causal inference is critical, but appropriately designed and analyzed observational studies can yield valid results that better balance internal and external validity than typical RCTs. When the question is one of translation and spread of innovations, a different set of questions comes into play: How and why does the intervention work? How can a model be amended or adapted to work in new settings? In these “delivery system science” settings, causal inference is not the main issue, so a range of quantitative, qualitative, and mixed research designs are needed.We then describe why RCTs are regarded as the gold standard for assessing cause and effect, how alternative approaches relying on observational data can be used to the same end, and how observational studies of EHD can be effective complements to RCTs. We also describe how RCTs can be a model for designing rigorous observational studies, building an evidence base through iterative studies that build upon each other (i.e., confirmation across multiple investigations).

Adapting an Interdisciplinary Learning Health System Framework for Academic Health Centers: A Scoping Review

“…32 Also, Friedman and Rigby (2013) noted that flexibility ensures goals are fluid “enough to allow the system to perform its key functions but not so much that over-standardization stifles innovation and growth.” 23 Several studies recommended the use of testing phases to achieve collaboration and foster mutual performance expectations. 4,6,22,47…”

“…Bietenbeck and Prasser (2018) described LHSs as “entities in which progress in science, informatics, and care culture align to generate new knowledge as an ongoing, natural by-product of the care experience.” 9 Forrest et al (2014) defined LHSs as “integrating research done in routine care settings, structured data capture during every encounter, and quality improvement processes to rapidly implement advances in new knowledge, all with active and meaningful patient participation.” 27 LHS definitions frequently included processes for incorporating evidence into practice, structuring data for practical use and research, and aligning research efforts with quality care. 4,23,27–31…”

“…An LHS, which is defined as a systematic process for aligning science, informatics, and clinical practice to integrate providers, researchers, and patients as active participants in an evidence-based care continuum, 4,5 can provide an ideal environment for academic medicine to rapidly adopt evidence-based guidelines and translate research into practice. 5,6 For an LHS to achieve its potential, organized and contextually relevant frameworks are needed to support implementation, reduce confusion, and develop effective collaborations.…”

Implementing learning health systems in the UK NHS: Policy actions to improve collaboration and transparency and support innovation and better use of analytics

“…In combination, these factors have contributed to a lack of specialist analytical capacity in the NHS . So while analytics methods already used for research and evaluation of health care could be applied in LHS, there is scarce capacity to do this in either the NHS or research communities. For LHS, this is a significant challenge, as methods used for robust analysis of large health datasets are not routinely undertaken in the NHS.…”

Clinic Factors Associated With Mailed Fecal Immunochemical Test (FIT) Completion: The Difference-Making Role of Support Staff

“…These findings are useful given a recent CCM analysis that identified the role of immutable clinic factors (eg, size, ownership) in changes in preventive care delivery. 54 Our approach identified modifiable factors that individual clinics or learning health care systems 57,58 can address to improve FIT screening. Although systems might have standardized protocols and centralized programs to support population outreach, cliniclevel activities also contribute to program effectiveness.…”