Development of the Individual Participant Data (IPD) Integrity Tool for assessing the integrity of randomised trials using individual participant data

Hunter, KE; Aberoumand, M; Libesman, S; Sotiropoulos, JX; Williams, J; Li, W; Aagerup, J; Mol, BW; Wang, R; Barba, A; Shrestha, N; Webster, AC; Seidler, AL

doi:10.1101/2023.12.11.23299797

Cited by 5 publications

References 39 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

Initial Oxygen Concentration for the Resuscitation of Infants Born at Less Than 32 Weeks’ Gestation

Sotiropoulos,

Oei,

Schmölzer

et al. 2024

JAMA Pediatr

View full text Add to dashboard Cite

ImportanceResuscitation with lower fractional inspired oxygen (FiO2) reduces mortality in term and near-term infants but the impact of this practice on very preterm infants is unclear.ObjectiveTo evaluate the relative effectiveness of initial FiO2 on reducing mortality, severe morbidities, and oxygen saturations (SpO2) in preterm infants born at less than 32 weeks’ gestation using network meta-analysis (NMA) of individual participant data (IPD).Data SourcesMEDLINE, Embase, CENTRAL, CINAHL, ClinicalTrials.gov, and WHO ICTRP from 1980 to October 10, 2023.Study SelectionEligible studies were randomized clinical trials enrolling infants born at less than 32 weeks’ gestation comparing at least 2 initial oxygen concentrations for delivery room resuscitation, defined as either low (≤0.3), intermediate (0.5-0.65), or high (≥0.90) FiO2.Data Extraction and SynthesisInvestigators from eligible studies were invited to provide IPD. Data were processed and checked for quality and integrity. One-stage contrast-based bayesian IPD-NMA was performed with noninformative priors and random effects and adjusted for key covariates.Main Outcomes and MeasuresThe primary outcome was all-cause mortality at hospital discharge. Secondary outcomes were morbidities of prematurity and SpO2 at 5 minutes.ResultsIPD were provided for 1055 infants from 12 of the 13 eligible studies (2005-2019). Resuscitation with high (≥0.90) initial FiO2 was associated with significantly reduced mortality compared to low (≤0.3) (odds ratio [OR], 0.45; 95% credible interval [CrI], 0.23-0.86; low certainty) and intermediate (0.5-0.65) FiO2 (OR, 0.34; 95% CrI, 0.11-0.99; very low certainty). High initial FiO2 had a 97% probability of ranking first to reduce mortality. The effects on other morbidities were inconclusive.Conclusions and RelevanceHigh initial FiO2 (≥0.90) may be associated with reduced mortality in preterm infants born at less than 32 weeks’ gestation compared to low initial FiO2 (low certainty). High initial FiO2 is possibly associated with reduced mortality compared to intermediate initial FiO2 (very low certainty) but more evidence is required.

show abstract

Initial Oxygen Concentration for the Resuscitation of Infants Born at Less Than 32 Weeks’ Gestation

Sotiropoulos,

Oei,

Schmölzer

et al. 2024

JAMA Pediatr

View full text Add to dashboard Cite

show abstract

The Individual Participant Data Integrity Tool for assessing the integrity of randomised trials

Hunter,

Aberoumand,

Libesman

et al. 2024

Research Synthesis Methods

View full text Add to dashboard Cite

Increasing concerns about the trustworthiness of research have prompted calls to scrutinise studies' Individual Participant Data (IPD), but guidance on how to do this was lacking. To address this, we developed the IPD Integrity Tool to screen randomised controlled trials (RCTs) for integrity issues. Development of the tool involved a literature review, consultation with an expert advisory group, piloting on two IPD meta‐analyses (including 73 trials with IPD), preliminary validation on 13 datasets with and without known integrity issues, and evaluation to inform iterative refinements. The IPD Integrity Tool comprises 31 items (13 study‐level, 18 IPD‐specific). IPD‐specific items are automated where possible, and are grouped into eight domains, including unusual data patterns, baseline characteristics, correlations, date violations, patterns of allocation, internal and external inconsistencies, and plausibility of data. Users rate each item as having either no issues, some/minor issue(s), or many/major issue(s) according to decision rules, and justification for each rating is recorded. Overall, the tool guides decision‐making by determining whether a trial has no concerns, some concerns requiring further information, or major concerns warranting exclusion from evidence synthesis or publication. In our preliminary validation checks, the tool accurately identified all five studies with known integrity issues. The IPD Integrity Tool enables users to assess the integrity of RCTs via examination of IPD. The tool may be applied by evidence synthesists, editors and others to determine whether an RCT should be considered sufficiently trustworthy to contribute to the evidence base that informs policy and practice.

show abstract

Development of the individual participant data integrity tool for assessing the integrity of randomised trials using individual participant data

Hunter,

Aberoumand,

Libesman

et al. 2024

Research Synthesis Methods

View full text Add to dashboard Cite

Increasing integrity concerns in medical research have prompted the development of tools to detect untrustworthy studies. Existing tools primarily assess published aggregate data (AD), though scrutiny of individual participant data (IPD) is often required to detect trustworthiness issues. Thus, we developed the IPD Integrity Tool for detecting integrity issues in randomised trials with IPD available. This manuscript describes the development of this tool. We conducted a literature review to collate and map existing integrity items. These were discussed with an expert advisory group; agreed items were included in a standardised tool and automated where possible. We piloted this tool in two IPD meta‐analyses (including 116 trials) and conducted preliminary validation checks on 13 datasets with and without known integrity issues. We identified 120 integrity items: 54 could be conducted using AD, 48 required IPD, and 18 were possible with AD, but more comprehensive with IPD. An initial reduced tool was developed through consensus involving 13 advisors, featuring 11 AD items across four domains, and 12 IPD items across eight domains. The tool was iteratively refined throughout piloting and validation. All studies with known integrity issues were accurately identified during validation. The final tool includes seven AD domains with 13 items and eight IPD domains with 18 items. The quality of evidence informing healthcare relies on trustworthy data. We describe the development of a tool to enable researchers, editors, and others to detect integrity issues using IPD. Detailed instructions for its application are published as a complementary manuscript in this issue.

show abstract

Development of the Individual Participant Data (IPD) Integrity Tool for assessing the integrity of randomised trials using individual participant data

Cited by 5 publications

References 39 publications

Initial Oxygen Concentration for the Resuscitation of Infants Born at Less Than 32 Weeks’ Gestation

Initial Oxygen Concentration for the Resuscitation of Infants Born at Less Than 32 Weeks’ Gestation

The Individual Participant Data Integrity Tool for assessing the integrity of randomised trials

Development of the individual participant data integrity tool for assessing the integrity of randomised trials using individual participant data

Contact Info

Product

Resources

About