The Structured Process to Identify Fit‐For‐Purpose Data: A Data Feasibility Assessment Framework

Abstract

To complement real-world evidence (RWE) guidelines, the 2019 Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent real-world Evidence (SPACE) framework elucidated a process for designing valid and transparent real-world studies. As an extension to SPACE, here, we provide a structured framework for conducting feasibility assessments-a step-by-step guide to identify decision grade, fit-for-purpose data, which complements the United States Food and Drug Administration (FDA)'s framework for a RWE program. The process was informed by our collective experience conducting systematic feasibility assessments of existing data sources for pharmacoepidemiology studies to support regulatory decisions. Used with the SPACE framework, the Structured Process to Identify Fit-For-Purpose Data (SPIFD) provides a systematic process for conducting feasibility assessments to determine if a data source is fit for decision making, helping ensure justification and transparency throughout study development, from articulation of a specific and meaningful research question to identification of fit-for-purpose data and study design. BACKGROUNDAccess to extensive and diverse real-world data (RWD) sources has grown exponentially over the past decade. [1][2][3] Receptivity to using RWD in real-world evidence (RWE) to complement clinical trial evidence has simultaneously increased, 4-7 leading to more frequent inclusion of RWD studies in regulatory and payer submission packages, 8,9 but with mixed success. Whereas particular therapeutic areas, such as oncology and rare diseases, have historically utilized RWE, advances are being made to understand the optimal settings for producing RWE fit for decision making by regulators, payers, and health technology assessment agencies. 10 Standardssuch as guidance documents, step-by-step processes, and templates, developed to guide researchers on the design and conduct of RWD studies-support validity and transparency, and ultimately bolster confidence in RWE. These good practices cover the continuum 11 from articulating a clear research question 12 to transparency in study conduct and reporting of results, [13][14][15][16] and include consideration of the hypothetical target trial, 12,17 identifying confounders by constructing causal diagrams, 12,18,19 identifying a fit-for-purpose design, 12,20 protocol development, [21][22][23][24][25][26][27] and visualizing the study design. 20 A Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent RWE (SPACE) framework elucidated a step-by-step process for designing valid and transparent real-world studies and provides templates to capture decision making and justification at each step. 12 The structured template for planning and reporting on the implementation of RWE studies (STaRT-RWE) picks up where SPACE leaves off, providing detailed templates to capture the final design and implementation details (e.g., specific algorithms for each study variable). Taken...

Visualizations throughout pharmacoepidemiology study planning, implementation, and reporting

“…23 Assessing whether the data are fit-for-purpose is contextual and contingent on the study's research question. 23 Following the steps laid out in Gatto et al 24 for identifying fit-for-purpose data, a CONSORT-like diagram 25,26 can show considered data sources, those included/excluded based on prespecified criteria (evaluated without assessing associations among treatments and outcomes of interest), and those chosen for more detailed feasibility assessment.…”

Profiling real‐world data sources for pharmacoepidemiologic research: A call for papers

“…Existing guidance documents on real‐world evidence rightfully emphasize the importance of the quality and fitness‐for‐purpose of the real‐world data, as the validity of evidence is highly dependent on the quality of the underlying data 23,24 . Experts recommend against evaluating the quality of data sources in isolation, but rather in the context of specific research questions 25–31 . A chosen data source is only appropriate or fit‐for‐purpose if it contains the information necessary to address the research question being examined.…”

“…23,24 Experts recommend against evaluating the quality of data sources in isolation, but rather in the context of specific research questions. [25][26][27][28][29][30][31] A chosen data source is only appropriate or fit-for-purpose if it contains the information necessary to address the research question being examined. Specifically, a fit-for-purpose data source should have high reliability (i.e., data represent the intended underlying medical concepts, demonstrating accuracy, completeness, provenance, and traceability) and relevance (i.e., data represent the population of interest and include critical data elements needed to answer the research question in the clinical context of interest).…”

Transferability of real-world data across borders for regulatory and health technology assessment decision-making

“…For instance, a retrospective analysis of single-arm HTA submissions reported that the proportion of submissions that included RWD-based external control arms (ECAs) increased 22% per year over 2015-2019 (6). In response to the growing use of RWD, there has also been a proliferation of RWE guidelines that provide important recommendations and tools for the proper conduct of RWE studies (7)(8)(9)(10)(11)(12)(13). A main focus of these guidelines is ensuring that RWD are "fit-forpurpose, " (9) meaning that the data are high-quality/reliable (i.e., accurate, complete, properly audited, and traceable) and relevant (i.e., contain key data elements, sufficient in sample size, and representative) to the research question (12).…”

“…In response to the growing use of RWD, there has also been a proliferation of RWE guidelines that provide important recommendations and tools for the proper conduct of RWE studies ( 7 – 13 ). A main focus of these guidelines is ensuring that RWD are “fit-for-purpose,” ( 9 ) meaning that the data are high-quality/reliable (i.e., accurate, complete, properly audited, and traceable) and relevant (i.e., contain key data elements, sufficient in sample size, and representative) to the research question ( 12 ).…”