Appropriate statistical methods for analysing trial data are critical for the correct interpretation of the results. Item 12 of the CONSORT statement (Box 1) relates to the statistical methods used in the reporting of trials, together with scientific and statistical principles concerning analyses of subgroups, endpoints and appropriate statistical tests. These issues need to be carefully considered before beginning a study and should be outlined in a standard trial protocol, which may be supplemented by a more extensive statistical analysis plan.1

Both primary and secondary endpoints should be clearly described in the objectives sections of the trial protocol.2 Statistical considerations appropriate to the design of the trial, including sample-size calculations, timelines for any interim analyses and a sketch of a proposed statistical plan for analysing these endpoints, should be detailed in the statistical section of the protocol and reported in subsequent publications.3 The analysis principle for the primary outcome must be that of intention-to-treat, where the data are analysed according to the treatment group to which they were randomised.4

Circumstances can arise where, after a trial commences, the primary outcome is deemed to be suboptimal. This most commonly occurs when the observed rate of the primary outcome is substantially lower than anticipated, reducing the ability (power) of the study to evaluate the effects of treatment on this outcome. This could be the result of a recent change in non-trial background therapy or to recruitment of a more healthy subset of the patients of interest. In these instances, it is possible to modify the primary outcome, provided the reason for so doing is not based on knowledge of interim results of the effect of treatment in the study. Thus, if study data indicate that the rate of myocardial infarction (the primary outcome) is much lower in the intervention or control arm than originally anticipated, it would be highly inappropriate to modify the primary outcome to, for example, include stroke, as this choice is potentially influenced by a knowledge of interim results of the effects of treatment in the study. However, using the overall event rate for myocardial infarction in the whole study cohort (blinded — not differentiated by treatment) could provide justification for endpoint modification in a valid way. Any change in primary outcome during the study requires careful thought, planning and documentation.

Analysis of the secondary outcomes needs to be described in the same way as that for the primary outcome, with sufficient documentation in the analysis plan as to how they will be analysed. Where possible, further exploratory analyses should be identified before the completion of the study, with a clear scientific rationale for the reason and value of such analyses.

It is essential that potential subgroup analyses are specified before the commencement of a study to guard against data "dredging" or "trawling". Applying many different statistical tests to the same data (eg, on subgroups or different outcomes) has the effect of greatly increasing the chance that at least one of these comparisons will be declared statistically significant even if there is no real difference. This practice is often termed data dredging.9 However, simply specifying a subgroup analysis in advance does not necessarily add scientific legitimacy to the interpretation. A number of strategies exist to ensure the credibility of subgroup analyses, and a checklist proposed by Simes (personal communication) suggests that the following criteria should be satisfied.

That there is a biological rationale for considering the subgroup separately from the rest of the patients in the study. Lack of strong biological or clinical evidence for why the treatment should have different effects in a particular subgroup would detract from support of a true underlying differential effect, even if a conventionally significant difference were found.

That there is prior evidence or belief that a differential treatment effect in a subgroup is plausible. Lack of prior evidence suggests that differential treatment effects observed in subgroups become hypothesis-generating observations rather than firm conclusions.

That there is statistical evidence (ie, a significant interaction) of a difference in the effect of treatment for the subgroup in question compared with the other patients. For example, if there is an apparent advantage of treatment in younger compared with older patients, then careful (clinical and statistical) examination of this difference is required before it can be confidently concluded that a true differential treatment benefit exists in the subgroup of younger patients. Studies are frequently underpowered to detect such interaction effects; nevertheless, lack of statistical evidence of such interaction should prohibit firmly concluding any differential treatment effect in the subgroup.

That there is independent confirmation from other factors in the study of the possible differential treatment effect in the subgroup. For example, if, in a trial examining the effect of chemotherapy in gastric cancer, it is observed that women survive longer after an intervention than men, supporting evidence could be to observe that the response rate to treatment was higher and time to disease progression was also longer in women compared with men.

Common pitfalls with subgroup analysis are focusing on the size of the P value and of the treatment effect in any subgroup, ignoring the play of chance. Other issues, such as the total number of subgroups examined, also play a major role in determining the credibility of any observed differential subgroup effect. Subgroups defined before initiating the study would be more credible in terms of true differences in effect on the study findings than those determined only at the time of analysis.