Generalising the results of trials to clinical practice

Randomised controlled trials should be the basis for developing clinical guidelines and for decisions about individual patient management. They should also inform public health policy. However, their capacity to fulfil these roles will depend on how closely a trial’s participants reflect the general population of patients with the disorder that has been investigated. The extent to which a trial’s findings are relevant to the broader population of patients with the disorder is referred to as the trial’s generalisability, or external validity. The CONSORT statement refers to generalisability under Item 21 (Box 1).1 Well-written reports should discuss the various factors that influence the generalisability of the trial’s findings. Julian and Pocock have proposed a checklist of questions to assist with this assessment (see Box 2).2

To determine the generalisability of a trial’s findings, several aspects require scrutiny.

Secondly, the baseline data should describe the population that participated in the trial. Demographic variables, age range, as well as clinical data such as blood pressure, staging of disease and any listed comorbidities, will help readers decide whether the trial population closely resembles the patient population (or individual patient) for which a decision about management is required. In some trials, the entry criteria are considerably broader than the population actually recruited. This discrepancy will only be evident if adequate baseline data are presented. For example, in a study of combination chemotherapy in malignant breast cancer, no radiotherapy was allowed in the protocol.3 Results were reported on the basis of the extent of lymph node involvement — 0–3 nodes or 4 or more — and readers might assume that the findings of the trial would apply to participants with many (more than 10) involved nodes. However, less than 8% of patients with more than 10 involved nodes were included in the study, as clinicians referred these higher-risk patients for radiotherapy rather than enrolling them in the trial.4 The original trial report did not tell readers that the group with more than 4 involved nodes actually comprised patients with primarily 4 to 10 involved nodes.5

If the population randomly allocated to groups within the trial represents only a small proportion of those with the condition of interest and assessed for eligibility, it is probable that the generalisability of the findings of the study will be limited. Large trials assessing warfarin therapy for atrial fibrillation have enrolled only about 15% of those who were potentially eligible, and this substantially limits the generalisability of their results.6,7

Where eligible patients who entered randomised trials have been compared with those who were eligible but did not participate, differences have emerged. In a study of therapy for temporomandibular disorders, 18 eligible patients did not consent and 60 were randomly allocated to trial arms.8 The 18 patients who did not consent reported more pain than those who participated, perhaps restricting the findings of the trial to those with milder pain. Differences were also evident between enrolled and unenrolled patients in the Thrombolysis in Myocardial Infarction (TIMI 9) trial.9 The TIMI 9 registry prospectively evaluated patients with ST-segment-elevation myocardial infarction. There were no exclusion criteria for the registry, but there were exclusion criteria for the randomised trial. Patients in the registry, but not enrolled in the trial, had higher baseline risk for adverse outcomes.

In some trials, during random allocation, patients may be stratified by comorbidities regarded as potential confounders. In others, particularly in clinical trials of new drugs, comorbidities may be exclusion criteria. If these comorbidities are relatively common, the exclusion criteria will significantly limit the generalisability of the trial outcomes. This is an important issue in drug trials, as comorbidities are often exclusion criteria. When the drug is registered for use, the listed indication is often relatively broad, so it is necessary to scrutinise the clinical trials section of the product information to obtain a clearer picture of the patient population which was studied. If a patient for whom the drug is being considered has one of the comorbidities which was an exclusion criterion, whether the drug will be efficacious or safe is unknown. Such a dilemma exists with the “statin” lipid-lowering drugs. Over 160 000 patients have participated in trials of statins, but almost all of these trials have excluded patients with significant renal or hepatic disease.11-13

If some patients have characteristics which were not reported in the trial’s population, it is conceivable that the trial’s results are not relevant to these patients.

As adolescents and pregnant women are not usually included in trials, most clinical trials are of limited relevance to these groups. If there are a priori reasons to expect differences between subgroups, the trial may have stratified participants by these potential confounders. The findings of the trial will be most generalisable if the benefits are evident in each subgroup of the trial, as well as across the entire study population.14 In clinical trials with large numbers of patients or events, it is possible to have reliable subgroup analyses which may help prescribers to relate the trial’s findings more closely to patients for whom they are trying to select appropriate therapies.

Numerous exclusion criteria will restrict the patient population and progressively diminish the generalisability of the findings of the intervention under evaluation. It is the responsibility of those reporting trials to include aspects of generalisability when discussing their findings. It is the task of those responsible for treating patients, producing clinical guidelines and formulating public health policy to carefully assess the generalisability of clinical trials before applying their findings.15