Study design flaws and statistical challenges in evaluating fertility treatments

Abstract

Health interventions should be tested before being introduced into clinical practice, to find out whether they work and whether they are harmful. However, research studies will only provide reliable answers to these questions if they are appropriately designed and analysed. But these are not trivial tasks. We review some methodological challenges that arise when evaluating fertility interventions and explain the implications for a non-statistical audience. These include flexibility in outcomes and analyses; use of surrogate outcomes instead of live birth; use of inappropriate denominators; evaluating cumulative outcomes and time to live birth; allowing each patient or couple to contribute to a research study more than once. We highlight recurring errors and present solutions. We conclude by highlighting the importance of collaboration between clinical and methodological experts, as well as people with experience of subfertility, for realising high-quality research. Lay summary We do research to find out whether fertility treatments are beneficial and to make sure they don’t cause harm. However, research will only provide reliable answers if it is done properly. It is not unusual for researchers to make mistakes when they are designing research studies and analysing the data that we get from them. In this review, we describe some of the mistakes people make when they do research about fertility treatments and explain how to avoid them. These include challenges which arise due to the large number of things that can be measured and reported when looking to see if fertility treatments work; failure to check whether the treatment increases the number of live births; failing to include all study participants in calculations;challenges in studies where participants may have more than one treatment attempt. We conclude by highlighting the importance of collaboration between clinical and methodological experts, as well as people with experience of fertility problems.

Can Dietary Patterns Impact Fertility Outcomes? A Systematic Review and Meta-Analysis

“…Other limitations, contributing to significant heterogeneity, include variability in clinical populations and diets, both geographically and culturally, as well as in their methodology. These concerns and their implications have already been highlighted in other research in the area [49]. We also note limitations inherent to FFQs such as risk of recall and social approval bias [50].…”

Interpregnancy Interval After Clinical Pregnancy Loss and Outcomes of the Next Frozen Embryo Transfer

“…That is, women who are able to conceive naturally sooner after a preceding CPL (ie, shorter IPI) may have better fecundity, whereas women with a longer IPI are more likely to experience compromised fecundity or subfertility, resulting in a lower likelihood of achieving live birth that possibly counteracts the benefit of longer IPIs on pregnancy outcomes. 18 Although the results remained unaltered when taking into account the duration of trying to conceive, 14,19 there may have been a possibility of self-reporting bias or recall bias (eg, the estimated date of the last menstrual period or the recall of the duration couples spent trying to conceive). In addition, we observed a higher risk of biochemical pregnancy loss among women with an IPI of less than 3 months compared with those with an IPI of 6 to 12 months.…”

“…In most of the previous studies in naturally conceiving populations, IPI length was generally defined as the period between the end of the preceding CPL and the onset of the last menstrual cycle of the next pregnancy, which was likely confounded by female fecundity. That is, women who are able to conceive naturally sooner after a preceding CPL (ie, shorter IPI) may have better fecundity, whereas women with a longer IPI are more likely to experience compromised fecundity or subfertility, resulting in a lower likelihood of achieving live birth that possibly counteracts the benefit of longer IPIs on pregnancy outcomes . Although the results remained unaltered when taking into account the duration of trying to conceive, there may have been a possibility of self-reporting bias or recall bias (eg, the estimated date of the last menstrual period or the recall of the duration couples spent trying to conceive).…”

Ethical Issues of Artificial Intelligence & Assisted Reproductive Technologies

“…Their value, however, depends on the study design, and the pitfalls of fertility-related study designs have been well outlined in the literature. 154 Moreover, reviewers may find evaluating study designs difficult and a critical appraisal framework for this type of research may be needed. 155 One also needs to address questions of authorship and conflicts of interest such as whether simply providing access to a dataset would be sufficient to be listed as an author, and the possibility of conflicts of interest, 156 such as when an executive of an IVF clinic hands over access to medical records in exchange for pecuniary gain such as start-up shares.…”