Evolution not revolution: the future of the randomised controlled trial in intensive care research

Innovative design methodologies may improve the statistical efficiency and success of future randomised trials

Over the past two decades, hundreds of parallel arm, double blind and open label randomised trials have been conducted in the critically ill. The Australian and New Zealand Intensive Care Society (ANZICS) Clinical Trials Group (CTG) has been at the forefront, conducting pivotal trials that have changed international practice in fields such as fluid resuscitation,1,2 sepsis,3,4 renal failure,5 traumatic brain injury6 and nutrition7. To date, only a handful of multicentre randomised trials have demonstrated improved patient outcomes.8,9 The initial promise of interventions such as protocolised, goal‐directed haemodynamic resuscitation for early septic shock (early goal‐directed therapy),3,10 tight glycaemic control11,12 and recombinant human activated protein C for severe sepsis13 has not been reproduced in subsequent confirmatory trials. Subsequent debate over the role of randomised trials to inform intensive care practice has led to calls for them to be abandoned,14 with others stating that they are “doomed to fail”.15 Is such a revolution in research methodology required? Or is it possible for the randomised clinical trial to evolve? A summary of studies discussed in this article is provided in the Box.

Despite the lack of clear improvement in patient outcomes, the impact of neutral trials on clinical practice cannot be underestimated. A critical examination of secondary outcomes, subgroup analyses, adverse events, resource utilisation and cost‐effectiveness is essential before disregarding trials with a null effect. While the primary outcome of CHEST (the Crystalloid versus Hydroxyethyl Starch Trial) (90‐day mortality) was not different between patients randomised to 6% hydroxyethyl starch or 0.9% saline, the need for renal replacement therapy (a secondary outcome) increased following hydroxyethyl starch administration.1 This finding contributed to the worldwide decrease in the use of hydroxyethyl starch for fluid resuscitation.22 International sepsis guidelines recommend 4% albumin (in addition to crystalloids) for resuscitation as a result of predefined subgroup analysis in the SAFE (Saline versus Albumin Fluid Evaluation) trial, which suggested a mortality benefit with 4% albumin administration for patients with severe sepsis.23 Conversely, albumin administration was associated with harm in the traumatic brain injury subgroup.24 While the primary outcome (90‐day mortality) in both TARGET (The Augmented versus Routine Approach to Giving Energy Trial) and the Sedation Practice in Intensive Care Evaluation: III trial was not different between treatment groups, increased complications in the intervention arms suggest that caution should be exercised with these therapies.7,16 Finally, ARISE (Australasian Resuscitation In Sepsis Evaluation) and other phase 3 early goal‐directed therapy trials reported increased resource use (intensive care admission, vasopressor administration, blood transfusion) without any improvement in survival or functional outcomes up to one year after randomisation with subsequent changes to guidelines for the haemodynamic resuscitation of septic shock.3,17

Clinical trials conducted in the critically ill have led to significant benefits to the Australian health care system. In 2017, the Australian Clinical Trials Alliance, on behalf of the Australian Commission on Safety and Quality in Health Care, undertook an economic evaluation of investigator‐initiated clinical trials conducted by Australian networks.25 The report found that the overall consolidated benefit–cost ratio was a return of $5.80 for every $1 invested, and for every $1 awarded in National Health and Medical Research Council grants, a return of $51.10 was achieved. CHEST, SAFE, ARISE and four other investigator‐initiated randomised clinical trials conducted by the ANZICS CTG were included in the analysis, with an estimated gross benefit of $271 million, of which $95 million was derived from avoidance of direct service costs and $176 million through better health outcomes. None of the included CTG trials demonstrated clinical superiority with the tested intervention. The report clearly demonstrates that large scale, multicentre, randomised trials are a worthwhile societal investment, leading to improved patient outcomes and reduced health care costs. The real questions are, given the substantial efforts required for successful trial completion, how we best achieve the greatest return, and whether there are there alternative methods of trial design.

First, we need to ensure that randomised trials are conducted according to rigorous methodological standards. Only then will the results demand attention rather than generate unnecessary argument regarding the trial's methodology. Integrity of the randomisation process must be ensured, along with concealment of group allocation, blinding (wherever feasible) and maximum data capture for all randomised participants. Newer initiatives such as mandatory trial registration and pre‐specified, pre‐published statistical analysis plans also help to minimise selective analysis and reporting bias.

Second, trial design should be appropriate for the study question and the intervention being tested. Evaluation of drug A versus drug B is typically conducted using the traditional parallel arm, double blind trial design with individual patient randomisation. Recent critical care examples include the ADRENAL (Adjunctive Corticosteroid Treatment in Critically Ill Patients with Septic Shock)4 and TARGET7 trials. Process of care interventions, such as medical emergency teams, may be better suited to a cluster or cluster crossover design with randomisation occurring at the site or hospital level. However, cluster trial designs may have reduced statistical efficiency relative to trials that randomise a similar number of individuals. This design feature is related to the extent of heterogeneity within clusters. The Medical Emergency Response and Intervention Trial is one example that did not demonstrate an improvement in the primary outcome; however, the trial's sample size calculation was based on an intra‐cluster correlation coefficient that substantially misjudged the extent of variation within clusters.18

Third, it is imperative that researchers use realistic estimates to inform their sample size calculations. These include an estimate of the control arm mortality, the biologically plausible treatment effect size and the minimum clinically important difference. Overly enthusiastic estimates of effect size, known as delta inflation, can lead to an under‐estimation of the required sample size, threatening a trial's internal validity. Among 33 intensive care unit‐based randomised trials conducted between 2007 and 2013 and published in high impact journals, the observed control group mortality differed by more than 7.5% from the expected or hypothesised mortality in 22 trials (67%).26 Further, based on actual mortality rate, 21 trials (64%) were only powered to detect an absolute reduction of 10% or greater. So, what is a plausible effect size? In the ADRENAL,4 TARGET7 and Stress Ulcer Prophylaxis in the Intensive Care Unit19 trials, the hypothesised absolute mortality risk reduction ranged from 3.4% to 5%. The subsequent observed difference was approximately 1% in all three trials. Trials powered to detect small but clinically important differences have been performed in other disciplines.27,28 The Clinical Randomisation of an Antifibrinolytic in Significant Haemorrhage 2 trial of tranexamic acid in bleeding trauma patients recruited over 20 000 patients for a hypothesised mortality benefit of only 2%.20 Using TARGET as an example, to detect a difference in outcome of 1%, 50 000 participants would be required to detect a treatment difference. Recruitment of very large patient numbers may be unrealistic with traditional trial designs in critical care. How then do we address the challenge of providing robust evidence that informs clinical practice?

Adaptive randomised clinical trials that are embedded into routine clinical care, such as the CTG‐endorsed Randomised, Embedded Multifactorial, Adaptive Platform Community‐Acquired Pneumonia trial (https://clinicaltrials.gov/ct2/show/NCT02735707) which recently commenced in Australia and New Zealand, have the potential to improve trial efficiency. Adaptive trials apply a Bayesian approach to analysis that incorporates prior knowledge into the trial design, as opposed to traditional trials which typically employ a frequentist approach. Based on accumulating data over the course of an ongoing trial, randomisation is prioritised, with ineffective or harmful therapies dropped from the randomisation schedule (response‐adaptive randomisation). However, adaptive trials are more complex to design and analyse than traditional randomised trials.

Reliable sample size calculations are confounded by the between‐patient heterogeneity of many critically ill populations. Within any clinical trial there may be subgroups of patients who receive a benefit from or, conversely, are harmed by the intervention under investigation. Enrichment strategies to prospectively identify this heterogeneity in treatment effect have the potential to increase statistical efficiency, leading to a larger effect size and a smaller study population. Patient selection may be based on an individual's likelihood of responding to an intervention (predictive enrichment) or the likelihood of a disease‐related event (prognostic enrichment). In highly heterogeneous populations such as those with sepsis and adult respiratory distress syndrome, the identification of specific physiology, biomarker expression, disease characteristics and baseline risk of death have the potential to reduce patient heterogeneity and enhance the ability to demonstrate a treatment effect (if one exists). The recent identification of four clinical sepsis phenotypes (α, β, γ and δ) that correlate with host response patterns, mortality and other patient outcomes is an exciting development for the design and interpretation of future targeted trials and the integration of precision medicine into clinical practice.29 The authors reported that estimations of treatment effect by phenotype in the Protocolized Care for Early Septic Shock trial suggest that increasing the δ phenotype trial population (highest mortality risk) increases the likelihood of harm with early goal‐directed therapy, while inclusion of more patients with the α phenotype (lowest mortality risk) increases the chances of finding a mortality benefit.29

Clinical registries may also provide a mechanism for enhanced trial efficiency. The ANZICS CTG‐endorsed Proton Pump Inhibitors versus Histamine‐2 Receptor Blockers for Ulcer Prophylaxis Therapy in the Intensive Care Unit study is a multicentre, cluster randomised, crossover, registry trial.21 The primary data repository for this study was an established registry (ANZICS Adult Patient Database). This facilitated recruitment of over 25 000 participants from over 40 centres in a 2‐year period. Registry embedded trials offer great promise; however, questions remain regarding their data quality and external validity.30

Despite the paucity of randomised trials reporting a positive treatment effect, clinical practice continues to be informed by study results, undoubtedly contributing to the secular decline in mortality following critical illness. High quality, robust, pragmatic, randomised clinical trials will continue to improve the care of our patients. Adoption of novel trial designs and enrichment strategies that identify patients most likely to benefit, in combination with an explosion in electronic health data that can refine trial methodology and also provide complementary evidence, is likely to play an important role in evaluating therapeutic strategies where clinical equipoise exists, particularly for interventions that are expensive, resource intensive or widely used in broad patient populations.