Guidance from key organisations on exploring, confirming and interpreting subgroup effects of medical treatments: a scoping review

Abstract

ObjectivesWith the increasing interest in personalised medicine, the use of subgroup analyses is likely to increase. Subgroup analyses are challenging and often misused, possibly leading to false interpretations of the effect. It remains unclear to what extent key organisations warn for such pitfalls and translate current methodological research to detect these effects into research guidelines. The aim of this scoping review is to determine and evaluate the current guidance used by organisations for exploring, confirming and interpreting subgroup effects.DesignScoping review.Eligibility criteriaWe identified four types of key stakeholder organisations: industry, health technology assessment organisations (HTA), academic/non-profit research organisations and regulatory bodies. After literature search and expert consultation, we identified international and national organisations of each type. For each organisation that was identified, we searched for official research guidance documents and contacted the organisation for additional guidance.ResultsTwenty-seven (45%) of the 60 organisations that we included had relevant research guidance documents. We observed large differences between organisation types: 18% (n=2) of the industry organisations, 64% (n=9) of the HTA organisations, 38% (n=8) of academic/non-profit research organisations and 57% (n=8) of regulatory bodies provided guidance documents. The majority of the documents (n=33, 63%) mentioned one or more challenges in subgroup analyses, such as false positive findings or ecological bias with variations across the organisation types. Statistical recommendations were less common (n=19, 37%) and often limited to a formal test of interaction.ConclusionsAlmost half of the organisations included in this scoping review provided guidance on subgroup effect research in their guidelines. However, there were large differences between organisations in the amount and level of detail of their guidance. Effort is required to translate and integrate research findings on subgroup analysis to practical guidelines for decision making and to reduce the differences between organisations and organisation types.

A systematic review of subgroup analyses in randomised clinical trials in cardiovascular disease

“…In general, it can be assumed that working in accordance with the mentioned guidelines also contributes to a qualitative improvement in this regard. 5 , 23 , 24 …”

“…In general, it can be assumed that working in accordance with the mentioned guidelines also contributes to a qualitative improvement in this regard. 5,23,24 In addition, author guidelines for selected journals were examined in regard to explicit requirements or instructions for the presentation of subgroup analyses (see journal websites for references). Providing information about the implementation of subgroup analyses in (R)CTs, such as the pre-specification or methods used for hypothesis testing, should be seen as a mandatory requirement for authors according to these guidelines.…”

“… 2 , 5 , 18 , 21 Recommendations and clear guidelines for the implementation and publication of subgroup analyses in the context of RCTs are available, which aim at increasing the comparability, generalizability and error control of results. 23 – 27 As major outcomes of previous reviews, it was shown that subgroup analyses have been published more frequently for large (R)CTs and in the case of non-significant primary analyses. 1 , 2 , 5 , 16 , 17 Results of at least one subgroup analysis were reported from 61% to 70% of reviewed (R)CTs, 2 , 5 , 14 , 21 with a median of up to four reported subgroup analyses per trial.…”

Subgroup analysis in haematologic malignancies phase III clinical trials: A systematic review

“…However, confirmatory studies are generally never carried out and decisions in clinical practice are made despite this lack of information. On the other hand, completely discarding subgroup analysis findings is also a decision that has consequences and is especially controversial in situations with very high risks or costs that are difficult to assume, which are not uncommon in clinical practice 11 …”

Uncovering Individualised Treatment Effect: Evidence from Educational Trials

“…Previous researchers have cautioned against potential overinterpretation of effects for subgroups like FSM students [17][18][19][20] . Although some challenges of subgroup analyses, such as lack of statistical power and unreliable estimation, can be partially alleviated by sound analysis and responsible reporting [17][18][19][20] , there are insufficient statistical details in the guidance of key organisations on how to exactly estimate, report, and interpret subgroup effects 21 .…”