Reporting and handling missing outcome data in mental health: a systematic review of Cochrane systematic reviews and meta-analyses

Abstract

Reporting of the techniques for handling missing outcome data and their implications in the findings of the systematic reviews are suboptimal.

Allowing for uncertainty due to missing and LOCF imputed outcomes in meta‐analysis

“…A standard methodology in many clinical fields for imputing incomplete longitudinal data sets is the last observation carried forward (LOCF) method: The missing outcome is replaced by the last observed value. Missing data are particularly evident in mental health trials where dropout rates may exceed 50% and the LOCF method is commonly applied …”

“…Although more appropriate methods have been proposed and adopted in new trials, older trials included in systematic reviews and meta‐analyses often use LOCF . A recent study showed that more than 75% of meta‐analyses in mental health contained studies that had LOCF imputed outcomes . The availability of individual participant data is rare and, as a result, meta‐analyses are not able to use appropriate imputation methods (eg, multiple imputation, likelihood methods) within each study.…”

Reporting guidance considerations from a statistical perspective: overview of tools to enhance the rigour of reporting of randomised trials and systematic reviews

“…Melander et al 10 found that there was considerable evidence of selective outcome reporting and selective publication based on inspection of 42 placebo-controlled trials of selective serotonin reuptake inhibitors provided to the Swedish drug regulatory authority, and suggested that any efforts to recommend a ‘best’ therapy in practice based on only public data are seemingly limited by biased data. Regarding the completeness of reporting of systematic reviews, Spineli et al 11 conducted a systematic review of Cochrane reviews to evaluate the extent to which they indicated methods to address missing study data and acknowledged their impact on the review, and found that in both cases there remains a need for improvement in transparency. There is an urgent need to address the challenge of poor reporting of many forms.…”

Missing binary data extraction challenges from Cochrane reviews in mental health and Campbell reviews with implications for empirical research

“…Eligible clinical trials of questionable internal validity due to inadequate reporting (ie, risk of bias is unclear for majority of domains) or poor design (ie, risk of bias is high for majority of domains) can greatly compromise the quality of inferences drawn from a systematic review, even if it has been conducted and reported rigorously, and according to current methodological standards of good practice (ie, PRISMA and Cochrane Handbook). In the case of missing outcome data, inadequate reporting of relevant aspects in publication (eg, outcome and number of patients who completed the trial) will challenge not only the handling of such data in meta‐analysis but also accuracy of data extraction and thus validity of meta‐analysis results …”

“…However, trial reports rarely distinguish between imputed and complete outcomes, and hence, the meta-analyst has no choice but to treat reported trial outcomes as being observed. 5 To the best of our knowledge, challenges of data extraction from meta-analyses in the context of missing outcome data have not been reported in the literature. Therefore, the present article aims to report, based on our recent experience with Cochrane systematic reviews, (1) the common challenges a researcher is likely to face Highlights • What is already known Missing outcome data are ubiquitous in systematic reviews of all research fields and pose challenges in their handling and interpretation.…”