In reply: A key message of our article is that the minimum possible difference that would render the intervention clinically worthwhile needs to be determined in the design phase of the study.1 This potential clinical difference (net advantage over standard care) must, of necessity, incorporate the potential trade-offs between any outcome advantages and associated toxicities and/or cost disadvantages.

In determining the minimum clinically worthwhile difference, the intention is not to ignore Level I evidence (phase III studies and meta-analyses) when such evidence exists. However, in most cases of designing a phase III study, such evidence is not available. In this instance, the use of phase II information (which can often provide good estimates of potential side effects and toxicities) can help to inform the estimated advantage in clinical outcome which would be needed to justify widespread use of the intervention. Tools have been developed to help clinicians determine worthwhile benefit against toxicity trade-offs for individual patients.2

While there are instances of study results reliably showing statistical significance without the measured effect being sufficiently large to be considered clinically relevant, this occurs rarely. Far more commonly, statistically non-significant results may obscure clinically relevant outcomes because studies have been seriously underpowered.3 For example, 17 of the first 22 small trials of thrombolytic therapy compared with placebo for acute myocardial infarction reported non-significant results, although they showed a 19% reduction in early mortality (2P < 0.01) in subsequent meta-analysis.4

Individual doctors treating their patients ultimately determine whether net differences are sufficiently worthwhile to change their clinical practice. The fundamental principle is that clinical trials should be designed with sufficient power (through having appropriate sample sizes) to detect differences that doctors would consider clinically worthwhile to improve health outcomes.