Using a meta-narrative literature review and focus groups with key stakeholders to identify perceived challenges and solutions for generating robust evidence on the effectiveness of treatments for rare diseases

Abstract

IntroductionFor many rare diseases, strong analytic study designs for evaluating the efficacy and effectiveness of interventions are challenging to implement because of small, geographically dispersed patient populations and underlying clinical heterogeneity. The objective of this study was to integrate perspectives from published literature and key rare disease stakeholders to better understand the perceived challenges and proposed methodological approaches to research on clinical interventions for rare diseases.MethodsWe used a meta-narrative literature review and focus group interviews with key rare disease stakeholders to better understand the perceived challenges in generating and synthesizing treatment effectiveness evidence, and to describe various research methods for mitigating these identified challenges. Data from both components of this study were synthesized narratively according to research paradigms that emerged from our data.ResultsResults from our meta-narrative literature review and focus group interviews revealed three fundamental challenges in generating robust treatment effectiveness evidence for rare diseases: i) limitations in recruiting a sufficient sample size to achieve planned statistical power; ii) inability to account for clinical heterogeneity and assess treatment effects across a clinical spectrum; and iii) reliance on short-term, surrogate outcomes whose clinical relevance is often unclear. We mapped these challenges and associated solutions to three interrelated research paradigms: i) explanatory evidence generation; ii) comparative effectiveness/pragmatic evidence generation; and iii) patient-oriented evidence generation. Within each research paradigm, numerous criticisms and potential solutions have been described with respect to overcoming these challenges from a research study design perspective.ConclusionsOver time, discussions about clinical research for interventions for rare diseases have moved beyond methodological approaches to overcome challenges related to explanatory evidence generation, with increased recognition of the importance of pragmatic and patient-oriented evidence. Future directions for our work include developing a framework to expand current evidence synthesis practices to take into consideration many of the concepts discussed in this paper.Electronic supplementary materialThe online version of this article (10.1186/s13023-018-0851-1) contains supplementary material, which is available to authorized users.

Outcomes in pediatric studies of medium-chain acyl-coA dehydrogenase (MCAD) deficiency and phenylketonuria (PKU): a review

“…Blood phenylalanine is a wellestablished surrogate indicator of clinical symptoms for PKU and has been used as a marker of treatment adherence in treatment guidelines and as a clinical trial outcome [13][14][15]. Industry-sponsored studies may also be more likely to incorporate short-term and surrogate outcomes in evaluating treatments for rare diseases [4], which could contribute to the prominence of pathophysiological end-points in studies of PKU. However, the relatively small number of articles on PKU that reported patient-oriented outcomes (as compared with blood phenylalanine) is potentially of concern.…”

“…Specifically, patient-oriented outcomes that reflect the lived experience of patients and their caregivers have emerged as a key priority for evaluative studies in the field of rare diseases [4]. The life impact core area, which covers many such outcomes, appeared relatively wellrepresented in articles reporting on MCAD deficiency, although most individual outcomes within this core area were themselves reported only once or twice, perhaps reflecting a lack of consensus on which aspects of life impacts are of highest priority for measurement.…”

“…Our search for relevant articles was extensive, covering electronic databases of peer reviewed literature, supplemented by a grey literature search guided by Grey Matters [20], additional search strategies for long-term follow-up initiatives of newborn screening, and a search of the COMET database for pediatric COSs [8]. Through data extraction and synthesis we created a comprehensive list of all of the outcomes reported or recommended Eye health 2 (1%) 0 (0%) 0 (0%) 1 (1%) 1 (1%) [1][2][3][4][5][6][7][8][9][10] indicates top ten most reported or discussed unique outcomes, ties indicated with an asterisks; p indicates neuro-psychological outcomes in the MCAD deficiency and PKU literature that can be used to support the development of COSs for these diseases. Our review also has limitations.…”

“…These diseases are typically diagnosed early in life, often involve complex and resource-intensive medical care [2], and are frequently associated with intense home management and caregiving needs [3]. Providing effective treatment can be difficult due to a scarcity of evidence supporting current therapies [4]. When properly conducted, randomized controlled trials are considered the 'gold standard' primary study design for evaluating interventions [5].…”

“…For the outcome, cognition and intelligence/IQ, there were 39 different instruments reported among 82 articles, with the Wechsler Intelligence Scales family of measurement instruments [23] being the most frequently specified (33/82 articles, 40%). Executive functioning was measured using 25 different instruments across 32 articles with the Behaviour Rating Inventory of General health biomarkers 2 (4%) 0 (0%) 0 (0%) 2 (7%) [1][2][3][4][5][6][7][8][9][10] indicates top ten most reported or discussed unique outcomes, ties indicated with an asterisks Child psychosocial well-being and self-concept 15 (4%) 0 (0%) 0 (0%) 10 (8%) 5 (7%)…”