Determining the sample size in a clinical trial

The minimum information needed to calculate sample size for a randomised controlled trial in which a specific event is being counted includes the power, the level of significance, the underlying event rate in the population under investigation and the size of the treatment effect sought. The calculated sample size should then be adjusted for other factors, including expected compliance rates and, less commonly, an unequal allocation ratio.

Power: The power of a study is its ability to detect a true difference in outcome between the standard or control arm and the intervention arm. This is usually chosen to be 80%. By definition, a study power set at 80% accepts a likelihood of one in five (that is, 20%) of missing such a real difference. Thus, the power for large trials is occasionally set at 90% to reduce to 10% the possibility of a so-called "false-negative" result.

Level of significance: The chosen level of significance sets the likelihood of detecting a treatment effect when no effect exists (leading to a so-called "false-positive" result) and defines the threshold "P value". Results with a P value above the threshold lead to the conclusion that an observed difference may be due to chance alone, while those with a P value below the threshold lead to rejecting chance and concluding that the intervention has a real effect. The level of significance is most commonly set at 5% (that is, P = 0.05) or 1% (P = 0.01). This means the investigator is prepared to accept a 5% (or 1%) chance of erroneously reporting a significant effect.

Underlying population event rate: Unlike the statistical power and level of significance, which are generally chosen by convention, the underlying expected event rate (in the standard or control group) must be established by other means, usually from previous studies, including observational cohorts. These often provide the best information available, but may overestimate event rates, as they can be from a different time or place, and thus subject to changing and differing background practices. Additionally, trial participants are often "healthy volunteers", or at least people with stable conditions without other comorbidities, which may further erode the study event rate compared with observed rates in the population. Great care is required in specifying the event rate and, even then, during ongoing trials it is wise to have allowed for sample size adjustment, which may become necessary if the overall event rate proves to be unexpectedly low.

Size of treatment effect: The effect of treatment in a trial can be expressed as an absolute difference. That is, the difference between the rate of the event in the control group and the rate in the intervention group, or as a relative reduction, that is, the proportional change in the event rate with treatment. If the rate in the control group is 6.3% and the rate in the intervention arm is 4.2%, the absolute difference is 2.1%; the relative reduction with intervention is 2.1%/6.3%, or 33%.

Estimating the plausible effect of treatment to be sought in a randomised controlled trial provides a further challenge, and may be the most common problem for reported trials. Too frequently, studies are designed to identify an implausibly large treatment effect (for example, a 30% to 50% reduction), when most important treatments that have been adopted into clinical practice have shown more modest benefits. When studies are designed to find unrealistically large reductions and fail, smaller real reductions are inevitably rendered statistically non-significant, leading to confusion about the value of the intervention studied. To resolve uncertainty, the study then needs to be repeated elsewhere, but with a larger sample size than before. Wherever possible, the minimum worthwhile difference in response should be determined from phase II or pilot studies and expert opinion from colleagues. Investigators should take into consideration any cost or logistical advantages or disadvantages of the interventional treatment compared with standard care.

A major limitation of many sample size calculations is the failure to account for patients' predictable lack of compliance with their allocated treatments. As compliance losses directly affect the size of the achievable treatment difference, they also affect the estimated sample size in a non-linear fashion. For example, a placebo-controlled study needing 100 patients per treatment arm, with 100% compliance, would require about 280 patients per arm if compliance is only 80% in each group (that is, 20% of patients allocated the investigational treatment fail to take it, and 20% of patients allocated to the placebo-control arm cross over to the investigational treatment). The compliance adjustment formula is adjusted n per arm equals N/([c1+c2–1]2), where c1 and c2 are the average compliance rates per arm (so, in the above example, adjusted n = 100/([0.8+0.8–1]2) = 280).

A one-to-one allocation to intervention and control treatment arms is the most common form of random allocation and results in the smallest sample size requirement. Sometimes different allocation ratios are chosen, resulting in a larger total sample size needed to achieve the same power. This may be justified where the investigational treatment is unusually expensive or complicated to administer.

The sample size calculation should be described in sufficient detail to allow its use in other protocols. The power, level of significance and the control and intervention event rates should be clearly documented. Information on the scheduled duration of the study, any adjustment for non-compliance and any other issues that formed the basis of the sample size calculation should be included. For continuous outcomes, in particular (eg, blood pressure), assumptions made about the distribution or variability of the outcome should be explicitly stated.