Invited editorial presents an accurate summary of the results of two randomised placebo-controlled trials of vertebroplasty

The editorial that we wrote at the request of the Editor of the Medical Journal of Australia1 presents an accurate summary of our two randomised placebo-controlled trials of vertebroplasty published in the New England Journal of Medicine.2,3 We discussed the negative findings of our high-quality trials in relation to the published literature, and explained why clinical experience and trials with inadequate experimental design can exaggerate treatment benefits.

Vertebroplasty has been used to treat painful osteoporotic vertebral fractures for 10 years, and has been publicly funded in Australia for over 4 years despite the absence of robust evidence to support its use. It is well accepted that biased uncontrolled observations and results from poorly designed studies can mislead decision making in health care, from treatment decisions for the individual patient to formulation of national public health policies.4 It might therefore be more appropriate for proponents of the procedure to stop searching for spurious reasons to dismiss our trial results and reflect on how it was possible for vertebroplasty to become the standard of care in the absence of appropriate evaluation in the first place.5

We have made our methods and results accessible to public scrutiny by publishing both the protocols and results in peer-reviewed journals. According to well established criteria,6 the results of both trials are at low risk of bias (Box 1). We have also scrupulously responded to unfounded criticisms of our trials made by Clark and others.10-12 Most if not all of the concerns raised by Clark et al in their critique in this issue of the Journal 13 are readily dismissed using an evidence-based framework and information publicly available in our published articles. Clark et al also present what may be pertinent observations based on their collective but unpublished experience of having treated 2500 patients with vertebroplasty.13 However, other than a quasi-experimental open controlled before–after observational study that compared the outcomes of volunteers who agreed to undergo vertebroplasty to those of patients who declined,7 and which was at high risk of bias (Box 1), they provide no data and scarce references in support of their assertions.

Diamond and Clark were valued collaborators throughout the implementation of the United States-based trial.3 Their enthusiasm for, and experience in, spine augmentation were key factors in allowing successful completion of that trial. Their plans to carry out a future similar trial in Sydney are laudable and it is to be hoped that this will add important information to our understanding about augmentation. It is well accepted that randomised placebo-controlled trials are the only truly valid means of evaluating the efficacy of treatment interventions, particularly when the condition being treated is self-limiting and the primary end point is improvement of symptoms.

As stated by Clark and colleagues,13 the natural history of acute osteoporotic vertebral fractures is generally favourable, with pain gradually subsiding over 6 to 12 weeks, while a minority present with more severe symptoms that often require opioid analgesia and sometimes hospitalisation. We also agree that those with more severe and persistent symptoms are likely to derive the most benefit from effective treatment.

Examination of the eligibility criteria and baseline characteristics of study populations in both trials confirm that these are the patients we studied. For example, nearly half the patients in both our US-based3 and Australian2 trials had overall or average pain severity scores of 8 or higher on a 0–10-point scale at baseline, and most participants in both trials were taking opioids (Box 2).

The study populations in both trials have been deemed by others to be typical of those seen in clinical practice.5,14 Our participants also shared comparable baseline characteristics, including levels of pain and disability, with participants in other vertebral augmentation trials, including both the VERTOS15 and FREE16 trials. Importantly, both trials included patients similar to those who would qualify for interim government-subsidised funding of the procedure in Australia — specifically, patients whose pain is not controlled by conservative medical therapy.17 Although the duration of medical therapy is not specified in this description of patients who qualify, historically, this has ranged from at least 4–6 weeks.

While Clark et al suggest that we enrolled patients very different from the ones they treat,13 other than a longer duration of symptoms in some participants, our trial population also appears similar with respect to baseline characteristics to patients in the study by Diamond et al7 (Box 2). Although participants in their study were reported to have acute symptoms (duration, 1 week to 6 weeks), no inclusion criteria based on symptom duration were specified and the mean duration of symptoms within each treatment group was not reported, making it unclear whether or not the treatment groups were comparable at baseline. It is not possible to compare baseline pain scores with the study by Diamond et al directly, as they assessed maximal pain for five activities to derive a score out of 25.7 However, it is likely that the participants in our trials had comparable or greater baseline levels of pain, as assessment of maximal pain with activity probably generates higher mean pain rating scores than assessments of resting, average or overall pain.18

We agree that the decision to proceed to vertebroplasty should be informed by both the clinical and radiological assessment of the patient. Contrary to the claims of Clark et al,13 an interventional radiologist was involved in confirming the eligibility of all participants in the Australian trial,2 and both outpatients and hospitalised patients were included.

There is no evidence that either trial was affected by selection bias. Participation rates of 36% and 30% of eligible patients in the Australian2 and US-based3 trials, respectively, are considered more than acceptable by usual trial standards, particularly considering both trials included a sham procedure. Furthermore, eligible patients who declined enrolment in the US-based trial3 had levels of pain and disability that were similar to those of the patients who participated.19

On the other hand, the study by Diamond et al7 was at high risk of selection bias (Box 1). The assignment of treatment was not random; it is well established that volunteers who agree to undergo intervention are likely to have better health outcomes than those who refuse (volunteer bias); and the participation rate was not reported in the original published report.7 In addition, a second publication reported the combined outcomes of the original study cohort as well as a further 47 patients which altered the results.8

While it is true that many patients will not consent to be randomly allocated into placebo-controlled trials, and the refusal rate may be higher for trials assessing invasive treatments, this will not affect the internal validity of the trial. Furthermore, the assertion that participants with severe pain would more often opt out of the trial and proceed with vertebroplasty is not supported by data from the Diamond et al study.7 When offered vertebroplasty, 30% of patients preferred usual medical care and their baseline pain levels were comparable to those of patients who agreed to undergo the procedure. This assertion is also without foundation for our Australian trial,2 because the availability of vertebroplasty outside of the trial was limited in our setting, particularly before government-subsidised funding. In fact, most of those who declined participation in the Australian trial did not wish to undergo any procedure, preferring to adopt a “wait-and-see” approach.

Current evidence does not support the claim made by Clark et al13 that the results of vertebroplasty are better in their hands and/or when the procedure is restricted to patients who have had symptoms for 6 weeks or less. The original study report by Diamond et al failed to demonstrate any difference in clinical outcomes between treatment groups at 6 weeks or at 6 – 12 months,7 and exploratory analysis of the approximately 20% of patients in the US-based trial3 who had fractures of less than 6 weeks’ duration did not find any evidence of benefit of vertebroplasty over placebo. There was also no evidence that symptom duration was a treatment effect modifier in our Australian trial.2

While we acknowledge that power was suboptimal for subgroup analysis when performed for both trials individually, the suggestion that vertebroplasty might be more effective for a subgroup of patients with very recent symptoms (ie, of less than 6 weeks’ duration) is improbable for several reasons. As Clark et al13 and others20 have outlined, most osteoporotic spinal fractures heal quickly; this implies that most people would be unlikely to benefit from early invasive intervention, and a subgroup with more persistent symptoms may be more likely to derive benefit. Secondly, the net overall effect of vertebroplasty in both trials was close to zero, making it unlikely that there would be subgroups that would benefit from the procedure.21 The only way that vertebroplasty could have a large effect on a proportion of patients would be if the condition of a substantial proportion were made worse — a scenario that is not reflected in the available data.10

Finally, as Clark et al acknowledge,13 the results of an open randomised controlled trial of vertebroplasty versus usual care that included only patients with very recent symptoms also failed to show any benefit of vertebroplasty over usual care at 3 months,22 further refuting the contention that benefits are more likely if the treatment is given early. While this study reported a significant reduction in pain at 24 hours after vertebroplasty,22 no comparative data were presented, so this finding provides no insight into the value of vertebroplasty.

As we have stated previously,10 the decision by one centre to withdraw from the Australian trial once government approval for reimbursement became available does not diminish the internal validity of the Australian trial.2 The other centre did not formally withdraw, and it contributed patients over the first 2 years. Clark et al also cast doubt on the validity of our trials by questioning the vertebroplasty techniques used in our trials.13 Both trials performed vertebroplasty according to standard methods, and we are not aware of any evidence to support claims made by Clark et al about cement volume or distribution.

Although neither trial reached their large pre-specified sample sizes — which were calculated to test hypotheses regarding safety, including the incident vertebral fracture rate associated with vertebroplasty in the Australian trial,2 and to detect very small differences in outcome in the US-based trial3 — both trials had more than adequate power to detect clinically important differences between groups with respect to their primary efficacy end points.5

As we stated in our editorial, anecdotal evidence and uncontrolled and open studies are well known to overestimate treatment benefit for many reasons, particularly when studying a self-limiting condition.1 In the case of vertebroplasty, reasons that the benefits of treatment could be overestimated include the favourable natural history of vertebral fractures; regression to the mean; the placebo effect; unblinded outcome assessment; and loss to follow-up, which is generally not random and is biased towards a more favourable outcome. The VERTOS II trial is an open randomised controlled trial that compares vertebroplasty to usual care.23 As such, it is also likely to have exaggerated treatment benefit for many of the reasons we have outlined.

Clark et al13 appear to suggest that a lower standard of evidence should apply for procedural medicine. New drug treatments are not considered to be of proven efficacy until, at the very least, they have been evaluated in randomised controlled trials. Even then, their safety is not guaranteed; in particular, adverse effects that are uncommon or those for which there is a long delay between exposure and clinical manifestation can occur. At the present time in Australia, it appears that new interventions are not always afforded the same degree of scrutiny as new drug therapies, although we would argue that, in the interests of our patients, the same onus of proof should be applied.

All patients should be honestly informed about the evidence for and against vertebroplasty. Not only is its efficacy unproven, the procedure poses some immediate, albeit small risks of cement leakage, infection and injury to the spinal cord, and it is not yet established whether or not it increases the risk of further vertebral fracture. Based on the results of our two trials that do provide the highest level of evidence we have to date on the value of vertebroplasty,2,3 routine use of this procedure outside of the research setting now appears unjustified. If there is any remaining uncertainty about the value of vertebroplasty for specific subgroups of patients, this should be addressed by further high- quality randomised placebo-controlled trials. In the meantime, primum non nocere, or first, do no harm.