Clinical trials represent a major investment by investigators, sponsors and participants, and it is reasonable to attempt to gain the maximum information from them. Practitioners and regulatory agencies are keen to know whether there are subgroups of trial participants who are more (or less) likely to be helped (or harmed) by the intervention under investigation, and a recent survey of trials published over 3 months in four leading journals found that 70% included subgroup analyses.1,2 Furthermore, regulatory guidance documents (such as the Committee for Proprietary Medicinal Products September 2002 document Points to consider on multiplicity issues in clinical trials3) strongly encourage appropriate subgroup analyses. The results of subgroup analyses can also drive changes in practice guidelines. For example, the United States National Institutes of Health issued a clinical alert following the unexpected finding in the BARI (Bypass Angioplasty Revascularisation Investigation) trial that mortality after angioplasty in patients with diabetes was nearly double that after bypass-graft surgery (P = 0.003).4

Meaningful information from subgroup analyses within a randomised trial is restricted by multiplicity of testing and low statistical power. There is therefore a tension between our wish to identify heterogeneity in the responses of trial participants to trial interventions and our technical capacity for doing so. Surveys on the adequacy of the reporting of clinical trials consistently find the reporting of subgroup analysis to be characterised by poor practice.2,5-7 Item 18 of the CONSORT checklist (Box 1) deals with the multiplicity issues that arise in subgroup analysis.8

The study report should include all the information required to assess the validity of subgroup analyses reported. In particular, the number of subgroup analyses should be declared, as this will enable readers to assess whether the issue of multiple testing is being dealt with. Analyses planned a priori, and the rationale for choosing them, should be clearly stated. Summary data, including event numbers and denominators for all the subgroup analyses, even the uninteresting ones, should be included, as this will facilitate future meta-analyses of the data and help prevent publication bias.18 The impact of multiple tests on the chance of declaring as statistically significant at least one false-positive result is shown in Box 3.

Some investigators avoid the issue of multiplicity of testing by tabulating the observed outcomes for the subgroups of interest without undertaking any formal statistical analysis. The data become available for meta-analysis,18 but there is the disadvantage that the investigator may fail to detect and draw attention to an important heterogeneity in the population.

The statistical methods used should be appropriate for the hypothesis being tested. The common practice of performing subgroup-specific tests of treatment effect is flawed in that it is testing the wrong hypothesis.19 The hypothesis that should be tested is whether the treatment effect in a subgroup is significantly different from that in the overall population.19 Testing for a statistically significant treatment effect in a subgroup is hindered by a small sample size.

The appropriate tests to use when analysing heterogeneity of responses among subgroups are interaction tests,2,10 for which worked examples are available.19,20 One study found that these were used in only 43% of 35 trials which reported subgroup analyses in their sample.2

Finally, the article should state whether the statistical tests used included adjustments for multiplicity.

Because subgroup analyses have less power to detect a therapeutic effect than the main study, the trial report, especially in the Abstract or Conclusions, should emphasise the overall result. Given the risks of false-positive findings when multiple subgroup analyses are performed, it is not surprising if a subgroup-specific test shows a significant (P < 0.05) or suggestive (P = 0.05 to P = 0.10) effect of treatment, even when the trial failed to do so overall.2,7

Investigators are often tempted to highlight a particular subgroup analysis.2,7 For example, in one trial the suggestion that a psychosocial nursing intervention following myocardial infarction was harmful for women (P = 0.064), but not men (P = 0.94), was highlighted, even though the intervention did not affect survival in the overall population21 (and a test for interaction was not significant2).

A number of arguments may be used to support the validity of a claimed subgroup effect (see, for example, the BARI trial4 and Rathore et al22):

• replication in another independent study;

• the presence of a dose–response relationship;

• reproducibilty of the observation in independent samples within the study, such as within individual sites; and

• the availability of a biological explanation.

Of these, the first is the strongest evidence. For example, even though the BARI study found no difference in survival following bypass surgery or angioplasty in the overall population, the validity of the subgroup findings was supported by other studies.4 On the other hand, the report by Rathore et al that digoxin use is associated with a significantly increased risk of death among women (P < 0.014)22 is weakened by the fact that it was a post-hoc analysis which was motivated by “biological suspicion” rather than by suggestive findings in earlier trials. Biological justifications for the findings of a posteriori (exploratory) analyses, on the other hand, carry little weight6,23 — the reports that diabetes is more common in boys born in October,24 and that lung cancer is more common in people born in March,25 included (in)credible biological explanations after the findings had been revealed.

The strategies for overcoming some of these difficulties in interpreting subgroup analyses will be explored in a forthcoming article in this series.