Evidence for compromised data integrity in studies of liberal peri‐operative inspired oxygen

Abstract

SummaryIn 2016 the World Health Organization recommended intra‐operative ventilation with 80% inspired oxygen to reduce surgical site infection rates, based upon a meta‐analysis of 15 randomised controlled trials, of which two were by Mario Schietroma's research group. Five trials by this group have been retracted for duplication, plagiarism, statistical error and lack of ethical approval. We analysed 40 papers by this group: 24 randomised controlled trials (5064 participants) and 16 observational studies (1847 patients). There was evidence that data integrity was compromised in 38 out of the 40 analysed papers. The distribution of baseline characteristics in randomised controlled trials was unlikely, p = 1.5 × 10−8: continuous variables within trials were heterogeneous, p = 1.9 × 10−9, and categorical variables were homogeneous, p = 8.5 × 10−20. Effects of interventions varied less than expected between studies: for categorical variables, for instance postoperative wound infection, p < 1 × 10−7, and for continuous variables, for instance HLA‐DR concentration, p = 0.00001. Of 184 calculable p values, for baseline variables or results, 179 (98%) were incorrect, ranging from three orders of magnitude too small to 10 orders of magnitude too large. Twenty‐one graphs occurred 81 times in 23 out of 40 papers. Liberal peri‐operative oxygen did not reduce surgical site infection in a meta‐analysis of 20 trials that excluded seven trials by Mario Schietroma and colleagues (odds ratio (95%CI) 0.89 (0.73‐1.08); p = 0.23). An update by the World Health Organization has now excluded trials of liberal oxygen by Schietroma's group, four of which have not been retracted. We conclude that Mario Schietroma's work should not inform practice until investigated.

Oxygen after surgery: review questions WHO advice that high levels reduce infection

“…Among the coauthors of the review published in Anaesthesia 2 is John Carlisle, a consultant anaesthetist at Torbay Hospital in Devon, UK, who has pioneered statistical studies of the baseline data of participants in trials that can show whether they are inconsistent with participants chosen randomly, as they should be. The team also checked the calculations of P values and found that a high proportion had been miscalculated—324 of 825.…”

Dissemination of medical publications on social media – is it the new standard?

“…Anaesthesia has an active social media presence, with initiatives including a regular ‘TweetChat’, where interested parties, including authors, are brought together on one platform to discuss a newly published paper . The recent #Hyperoxia TweetChat, on a paper about evidence for compromised data integrity in studies of liberal peri‐operative inspired oxygen , generated ~60,000 impressions (the number of times Tweets from @Anaes_Journal were seen) on the day of publication. Such initiatives must surely increase attention, citations and the speed with which the benefits from scientific research yield improvements in clinical care, but social media analytics is a new science and it is not yet clear how these metrics should be interpreted.…”

Data integrity issues: catalyst for a more robust approach to research on perioperative oxygen therapy?

“…In February 2019, data integrity from 38 published randomised controlled trials in surgical patients conducted by an Italian surgical research group was called into question (Buranyi and Devlin 2019). Using the Carlisle method, Myles, Carlisle and Scarr identified evidence of compromised data in 38 out of 40 published articles from Mario Schietroma’s research group spanning from years 1993 to 2016 (Myles et al 2019). In their analysis, they meta-analysed all published patient characteristic data, identifying poorly distributed participant characteristics, incorrect ‘ p ’ values and overly consistent results.…”

“…In their analysis, they meta-analysed all published patient characteristic data, identifying poorly distributed participant characteristics, incorrect ‘ p ’ values and overly consistent results. There was enough evidence from this analysis to recommend an inquiry into the group’s work, and their published outputs are likely to have been misleading researchers within the field of perioperative medicine for some years (Myles et al 2019). Such alleged misconduct is not only damaging to the scientific community and to perioperative research, but may also have substantial global consequences for patient safety through the impact of flawed research on doctors’ prescribing habits.…”