BackgroundDealing with heterogeneity in meta-analyses is often tricky, and there is only limited advice for authors on what to do. We investigated how authors addressed different degrees of heterogeneity, in particular whether they used a fixed effect model, which assumes that all the included studies are estimating the same true effect, or a random effects model where this is not assumed.MethodsWe sampled randomly 60 Cochrane reviews from 2008, which presented a result in its first meta-analysis with substantial heterogeneity (I2 greater than 50%, i.e. more than 50% of the variation is due to heterogeneity rather than chance). We extracted information on choice of statistical model, how the authors had handled the heterogeneity, and assessed the methodological quality of the reviews in relation to this.ResultsThe distribution of heterogeneity was rather uniform in the whole I2 interval, 50-100%. A fixed effect model was used in 33 reviews (55%), but there was no correlation between I2 and choice of model (P = 0.79). We considered that 20 reviews (33%), 16 of which had used a fixed effect model, had major problems. The most common problems were: use of a fixed effect model and lack of rationale for choice of that model, lack of comment on even severe heterogeneity and of reservations and explanations of its likely causes. The problematic reviews had significantly fewer included trials than other reviews (4.3 vs. 8.0, P = 0.024). The problems became less pronounced with time, as those reviews that were most recently updated more often used a random effects model.ConclusionOne-third of Cochrane reviews with substantial heterogeneity had major problems in relation to their handling of heterogeneity. More attention is needed to this issue, as the problems we identified can be essential for the conclusions of the reviews.

No abstract

GHG is co-chair of the GRADE working group and chaired the executive of 9th iteration of the American College of Chest Physicians Antithrombotic Guidelines. Description of session and speaker topics Session will comprise 3 presentations (15 mins), one for each challenge, with 5 mins for questions of clarification then 30 mins of panel discussion. Background Developing guidelines to inform decisions regarding diagnostic tests presents unique challenges that are not encountered when addressing intervention questions. In many cases, diagnostic studies only provide test accuracy results and lack patient outcomes; outcomes that are typically sought to make recommendations. Objectives/Goal Using the lessons from our guideline group, the objectives of this workshop are for participants to learn practical skills related to the development of guidelines for diagnostic questions. Specifically, the following areas will be addressed: Generating an appropriate research question. Developing relevant eligibility criteria for choosing diagnostic studies. Critically appraising diagnostic studies using existing tools and quality criteria. Determining what types of recommendations can be generated when different types of evidence and information are available and to respond when the most relevant information is not available. Target Group, Suggested Audience Guideline developers or anyone interested in how to develop a guideline for diagnostic questions. Description of the Workshop and of the Methods used to Facilitate Interactions Using a problem-based educational approach, the workshop will begin with a quick review of the background information and objectives, and an illustrative example will be presented. Participants will then be guided through the steps of guideline development for diagnostic questions, and given problems in each step to consider and work through in small groups. Finally, participants will develop recommendations for one or two guidelines, based on evidence from diagnostic guideline projects we have completed in our guideline group. INTERACTIVE WORKSHOPS 145WS EVIDENCE-BASED GUIDELINE DEVELOPMENT FOR DIAGNOSTIC QUESTIONS