Connect
MJA
MJA

Invited editorial presents an accurate summary of the results of two randomised placebo-controlled trials of vertebroplasty

Rachelle Buchbinder, Richard H Osborne and David Kallmes
Med J Aust 2010; 192 (6): 338-341.
Published online: 15 March 2010

The editorial that we wrote at the request of the Editor of the Medical Journal of Australia1 presents an accurate summary of our two randomised placebo-controlled trials of vertebroplasty published in the New England Journal of Medicine.2,3 We discussed the negative findings of our high-quality trials in relation to the published literature, and explained why clinical experience and trials with inadequate experimental design can exaggerate treatment benefits.

Vertebroplasty has been used to treat painful osteoporotic vertebral fractures for 10 years, and has been publicly funded in Australia for over 4 years despite the absence of robust evidence to support its use. It is well accepted that biased uncontrolled observations and results from poorly designed studies can mislead decision making in health care, from treatment decisions for the individual patient to formulation of national public health policies.4 It might therefore be more appropriate for proponents of the procedure to stop searching for spurious reasons to dismiss our trial results and reflect on how it was possible for vertebroplasty to become the standard of care in the absence of appropriate evaluation in the first place.5

We have made our methods and results accessible to public scrutiny by publishing both the protocols and results in peer-reviewed journals. According to well established criteria,6 the results of both trials are at low risk of bias (Box 1). We have also scrupulously responded to unfounded criticisms of our trials made by Clark and others.10-12 Most if not all of the concerns raised by Clark et al in their critique in this issue of the Journal 13 are readily dismissed using an evidence-based framework and information publicly available in our published articles. Clark et al also present what may be pertinent observations based on their collective but unpublished experience of having treated 2500 patients with vertebroplasty.13 However, other than a quasi-experimental open controlled before–after observational study that compared the outcomes of volunteers who agreed to undergo vertebroplasty to those of patients who declined,7 and which was at high risk of bias (Box 1), they provide no data and scarce references in support of their assertions.

Diamond and Clark were valued collaborators throughout the implementation of the United States-based trial.3 Their enthusiasm for, and experience in, spine augmentation were key factors in allowing successful completion of that trial. Their plans to carry out a future similar trial in Sydney are laudable and it is to be hoped that this will add important information to our understanding about augmentation. It is well accepted that randomised placebo-controlled trials are the only truly valid means of evaluating the efficacy of treatment interventions, particularly when the condition being treated is self-limiting and the primary end point is improvement of symptoms.

Patient selection

As stated by Clark and colleagues,13 the natural history of acute osteoporotic vertebral fractures is generally favourable, with pain gradually subsiding over 6 to 12 weeks, while a minority present with more severe symptoms that often require opioid analgesia and sometimes hospitalisation. We also agree that those with more severe and persistent symptoms are likely to derive the most benefit from effective treatment.

Examination of the eligibility criteria and baseline characteristics of study populations in both trials confirm that these are the patients we studied. For example, nearly half the patients in both our US-based3 and Australian2 trials had overall or average pain severity scores of 8 or higher on a 0–10-point scale at baseline, and most participants in both trials were taking opioids (Box 2).

The study populations in both trials have been deemed by others to be typical of those seen in clinical practice.5,14 Our participants also shared comparable baseline characteristics, including levels of pain and disability, with participants in other vertebral augmentation trials, including both the VERTOS15 and FREE16 trials. Importantly, both trials included patients similar to those who would qualify for interim government-subsidised funding of the procedure in Australia — specifically, patients whose pain is not controlled by conservative medical therapy.17 Although the duration of medical therapy is not specified in this description of patients who qualify, historically, this has ranged from at least 4–6 weeks.

While Clark et al suggest that we enrolled patients very different from the ones they treat,13 other than a longer duration of symptoms in some participants, our trial population also appears similar with respect to baseline characteristics to patients in the study by Diamond et al7 (Box 2). Although participants in their study were reported to have acute symptoms (duration, 1 week to 6 weeks), no inclusion criteria based on symptom duration were specified and the mean duration of symptoms within each treatment group was not reported, making it unclear whether or not the treatment groups were comparable at baseline. It is not possible to compare baseline pain scores with the study by Diamond et al directly, as they assessed maximal pain for five activities to derive a score out of 25.7 However, it is likely that the participants in our trials had comparable or greater baseline levels of pain, as assessment of maximal pain with activity probably generates higher mean pain rating scores than assessments of resting, average or overall pain.18

We agree that the decision to proceed to vertebroplasty should be informed by both the clinical and radiological assessment of the patient. Contrary to the claims of Clark et al,13 an interventional radiologist was involved in confirming the eligibility of all participants in the Australian trial,2 and both outpatients and hospitalised patients were included.

Selection bias

There is no evidence that either trial was affected by selection bias. Participation rates of 36% and 30% of eligible patients in the Australian2 and US-based3 trials, respectively, are considered more than acceptable by usual trial standards, particularly considering both trials included a sham procedure. Furthermore, eligible patients who declined enrolment in the US-based trial3 had levels of pain and disability that were similar to those of the patients who participated.19

On the other hand, the study by Diamond et al7 was at high risk of selection bias (Box 1). The assignment of treatment was not random; it is well established that volunteers who agree to undergo intervention are likely to have better health outcomes than those who refuse (volunteer bias); and the participation rate was not reported in the original published report.7 In addition, a second publication reported the combined outcomes of the original study cohort as well as a further 47 patients which altered the results.8

While it is true that many patients will not consent to be randomly allocated into placebo-controlled trials, and the refusal rate may be higher for trials assessing invasive treatments, this will not affect the internal validity of the trial. Furthermore, the assertion that participants with severe pain would more often opt out of the trial and proceed with vertebroplasty is not supported by data from the Diamond et al study.7 When offered vertebroplasty, 30% of patients preferred usual medical care and their baseline pain levels were comparable to those of patients who agreed to undergo the procedure. This assertion is also without foundation for our Australian trial,2 because the availability of vertebroplasty outside of the trial was limited in our setting, particularly before government-subsidised funding. In fact, most of those who declined participation in the Australian trial did not wish to undergo any procedure, preferring to adopt a “wait-and-see” approach.

Timing of vertebroplasty

Current evidence does not support the claim made by Clark et al13 that the results of vertebroplasty are better in their hands and/or when the procedure is restricted to patients who have had symptoms for 6 weeks or less. The original study report by Diamond et al failed to demonstrate any difference in clinical outcomes between treatment groups at 6 weeks or at 6 – 12 months,7 and exploratory analysis of the approximately 20% of patients in the US-based trial3 who had fractures of less than 6 weeks’ duration did not find any evidence of benefit of vertebroplasty over placebo. There was also no evidence that symptom duration was a treatment effect modifier in our Australian trial.2

While we acknowledge that power was suboptimal for subgroup analysis when performed for both trials individually, the suggestion that vertebroplasty might be more effective for a subgroup of patients with very recent symptoms (ie, of less than 6 weeks’ duration) is improbable for several reasons. As Clark et al13 and others20 have outlined, most osteoporotic spinal fractures heal quickly; this implies that most people would be unlikely to benefit from early invasive intervention, and a subgroup with more persistent symptoms may be more likely to derive benefit. Secondly, the net overall effect of vertebroplasty in both trials was close to zero, making it unlikely that there would be subgroups that would benefit from the procedure.21 The only way that vertebroplasty could have a large effect on a proportion of patients would be if the condition of a substantial proportion were made worse — a scenario that is not reflected in the available data.10

Finally, as Clark et al acknowledge,13 the results of an open randomised controlled trial of vertebroplasty versus usual care that included only patients with very recent symptoms also failed to show any benefit of vertebroplasty over usual care at 3 months,22 further refuting the contention that benefits are more likely if the treatment is given early. While this study reported a significant reduction in pain at 24 hours after vertebroplasty,22 no comparative data were presented, so this finding provides no insight into the value of vertebroplasty.

Vertebroplasty technique

As we have stated previously,10 the decision by one centre to withdraw from the Australian trial once government approval for reimbursement became available does not diminish the internal validity of the Australian trial.2 The other centre did not formally withdraw, and it contributed patients over the first 2 years. Clark et al also cast doubt on the validity of our trials by questioning the vertebroplasty techniques used in our trials.13 Both trials performed vertebroplasty according to standard methods, and we are not aware of any evidence to support claims made by Clark et al about cement volume or distribution.

Power and expected size of the treatment benefit

Although neither trial reached their large pre-specified sample sizes — which were calculated to test hypotheses regarding safety, including the incident vertebral fracture rate associated with vertebroplasty in the Australian trial,2 and to detect very small differences in outcome in the US-based trial3 — both trials had more than adequate power to detect clinically important differences between groups with respect to their primary efficacy end points.5

As we stated in our editorial, anecdotal evidence and uncontrolled and open studies are well known to overestimate treatment benefit for many reasons, particularly when studying a self-limiting condition.1 In the case of vertebroplasty, reasons that the benefits of treatment could be overestimated include the favourable natural history of vertebral fractures; regression to the mean; the placebo effect; unblinded outcome assessment; and loss to follow-up, which is generally not random and is biased towards a more favourable outcome. The VERTOS II trial is an open randomised controlled trial that compares vertebroplasty to usual care.23 As such, it is also likely to have exaggerated treatment benefit for many of the reasons we have outlined.

Conclusion

Clark et al13 appear to suggest that a lower standard of evidence should apply for procedural medicine. New drug treatments are not considered to be of proven efficacy until, at the very least, they have been evaluated in randomised controlled trials. Even then, their safety is not guaranteed; in particular, adverse effects that are uncommon or those for which there is a long delay between exposure and clinical manifestation can occur. At the present time in Australia, it appears that new interventions are not always afforded the same degree of scrutiny as new drug therapies, although we would argue that, in the interests of our patients, the same onus of proof should be applied.

All patients should be honestly informed about the evidence for and against vertebroplasty. Not only is its efficacy unproven, the procedure poses some immediate, albeit small risks of cement leakage, infection and injury to the spinal cord, and it is not yet established whether or not it increases the risk of further vertebral fracture. Based on the results of our two trials that do provide the highest level of evidence we have to date on the value of vertebroplasty,2,3 routine use of this procedure outside of the research setting now appears unjustified. If there is any remaining uncertainty about the value of vertebroplasty for specific subgroups of patients, this should be addressed by further high- quality randomised placebo-controlled trials. In the meantime, primum non nocere, or first, do no harm.

1 Comparison of the risk of bias of our randomised placebo-controlled trials and the Sydney-based observational study by Diamond and colleagues

Study feature

Buchbinder et al2

Kallmes et al3

Diamond et al7,8


Randomisation adequate?

Yes

Yes

Not randomised

Allocation concealed?

Yes

Yes

No

Groups similar at baseline?

Yes

Yes

Not reported for symptom duration, opioid analgesia, hospitalisation

Patients blinded?

Yes

Yes

No

Care providers blinded?

Yes

Yes

No

Outcome assessors blinded?

Yes

Yes

No

Sample size calculation described and acceptable?

Yes

Yes

Not reported

Primary end point specified?

Yes

Yes

No

Drop-out rate described and acceptable?

Yes

Yes

Not reported in primary article

Timing of outcome assessment similar?

Yes

Yes

Not reported for 6–12-month assessment (mean, 215 days; range, 57–399 days)

Intention-to-treat analysis?

Yes

Yes

Not reported in primary article

National Health and Medical Research Council (NHMRC) level of evidence (I highest to IV lowest)l9

II

II

III-2

2 Comparison of baseline characteristics of patients in the Buchbinder et al, Kallmes et al and Diamond et al vertebroplasty studies

Characteristic

Buchbinder et al2

Kallmes et al3

Diamond et al7,8


Sample size

78 (V, 38; P, 40)

131 (V, 68; P, 63)

79 (V, 55; UC, 24) 126 (V, 88; UC, 38; second article)*

Mean age (years)

V, 74.2; P, 78.9

V, 73.4; P, 73.3

V, 76.5; UC, 76.3

Female (%)

V, 82; P, 78

V, 78; P, 73

V, 64; UC, 83

Symptom duration (weeks)

V, 9; P, 9.5 (median)

V, 16; P, 20 (mean)

Not reported for total study sample or by treatment group

Proportion with short symptom duration

32%, < 6 weeks

20%, < 6 weeks; 41%, ≤ 13 weeks

100%, 1–6 weeks

Mean pain rating (SD)

V, 7.4 (2.1); P, 7.1 (2.3)

V, 6.9 (2.0);§ P, 7.2 (2.0)§

V, 19 (4); UC, 20 (5)

Modified Roland-Morris Disability score (0–23) (mean [SD])

V, 17.3 (2.8); P, 17.3 (2.9)

V, 16.6 (3.8); P, 17.5 (4.1)

Not measured

Proportion taking opioids at baseline

V, 79%; P, 85%

V, 56%; P, 63%

Not reported


V = vertebroplasty. P = placebo. UC = usual care. * Only baseline data from the original study are presented, as the second publication included an additional 47 patients. Participants reported to have a 1–6 week history of pain.  Overall pain over the course of the previous week on a scale of 0 (no pain) to 10 (maximum imaginable pain). § Average pain intensity during past 24 hours on a scale of 0 to 10, with higher scores indicating more severe pain.  Pain score out of 25 calculated by assessing pain associated with five activities: walking, climbing in and out of a chair, bathing, dressing, and resting, each on scales of 0 (no pain) to 5 (maximum pain; score recorded immediately on awakening, before administration of the morning dose of analgesia).

  • Rachelle Buchbinder1,2
  • Richard H Osborne3
  • David Kallmes4

  • 1 Monash Department of Clinical Epidemiology, Cabrini Hospital, Melbourne, VIC.
  • 2 Department of Epidemiology and Preventive Medicine, School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC.
  • 3 Public Health Innovation, Faculty of Health, Medicine, Nursing and Behavioural Sciences, Deakin University, Melbourne, VIC.
  • 4 Mayo Clinic, Rochester, Minn, USA.


Competing interests:

Rachelle Buchbinder and Richard Osborne were investigators for the Australian vertebroplasty trial,2 which was supported by grants from the National Health and Medical Research Council (NHMRC) (284354), Arthritis Australia, the Cabrini Education and Research Institute, and Cook Australia. David Kallmes was an investigator for the US-based trial,3 which was supported by a grant from the US National Institute of Arthritis and Musculoskeletal and Skin Diseases (R01-AR49373).

  • 1. Buchbinder R, Osborne R, Kallmes DF. Vertebroplasty appears no better than placebo for painful osteoporotic spinal fractures and has potential for harm [editorial]. Med J Aust 2009; 191: 476-477. <MJA full text>
  • 2. Buchbinder R, Osborne R, Ebeling P, et al. A randomized trial of vertebroplasty for painful osteoporotic vertebral fractures. N Engl J Med 2009; 361: 557-568.
  • 3. Kallmes D, Comstock B, Heagerty P, et al. A randomized trial of vertebroplasty for osteoporotic spinal fractures. N Engl J Med 2009; 361: 569-579.
  • 4. Altman D, Schulz K, Moher D, et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 2001; 134: 663-694.
  • 5. Carey TS. ACP Journal Club. Vertebroplasty was not effective for painful osteoporotic vertebral fractures. Ann Intern Med 2009; 151: 8-9.
  • 6. Higgins JPT, Green S (editors). Cochrane handbook for systematic reviews of interventions. Version 5.0.2 [updated September 2009]. The Cochrane Collaboration, 2009. http://www.cochrane-handbook.org (accessed Feb 2010).
  • 7. Diamond TH, Champion B, Clark WA. Management of acute osteoporotic vertebral fractures: a nonrandomized trial comparing percutaneous vertebroplasty with conservative therapy. Am J Med 2003; 114: 257-265.
  • 8. Diamond TH, Bryant C, Browne L, Clark WA. Clinical outcomes after acute osteoporotic vertebral fractures: a 2-year non-randomised trial comparing percutaneous vertebroplasty with conservative therapy. Med J Aust 2006; 184: 113-117. <MJA full text>
  • 9. National Health and Medical Research Council. NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Pilot program 2005–2007. Canberra: NHMRC, 2005. http://www.nhmrc.gov.au/_files_nhmrc/file/guidelines/levels_grades05.pdf (accessed Feb 2010).
  • 10. Trials of vertebroplasty for vertebral fractures [author reply]. New Engl J Med 2009; 361: 2099-2100.
  • 11. Buchbinder R, Kallmes D. Vertebroplasty: when randomized placebo-controlled trial results clash with common belief. Spine J 2010. In press.
  • 12. Kallmes D, Buchbinder R, Jarvik J, et al. Response to “Randomized vertebroplasty trials: bad news or sham news?” AJNR Am J Neuroradiol 2009; 30: 1809-1810.
  • 13. Clark WA, Diamond, TH, McNeil HP, et al. Vertebroplasty for painful acute osteoporotic vertebral fractures: recent Medical Journal of Australia editorial is not relevant to the patient group that we treat with vertebroplasty. Med J Aust 2010; 192: 334-337. <MJA full text>
  • 14. Kinkade S, Stevermer J. Vertebroplasty for osteoporotic fracture? Think twice. J Fam Pract 2009; 58: 654-656.
  • 15. Voormolen M, Mali W, Lohle P, et al. Percutaneous vertebroplasty compared with optimal pain medication treatment: short-term clinical outcome of patients with subacute or chronic painful osteoporotic vertebral compression fractures. The VERTOS Study. AJNR Am J Neuroradiol 2007; 28: 555-560.
  • 16. Wardlaw D, Cummings S, Van Meirhaeghe J, et al. Efficacy and safety of balloon kyphoplasty compared with non-surgical care for vertebral compression fracture (FREE): a randomised controlled trial. Lancet 2009; 373: 1016-1024.
  • 17. Medical Services Advisory Committee. Minutes 31st meeting, 24 August 2005, Brisbane. http://www.msac.gov.au/internet/msac/publishing.nsf/Content/minutes-1 (accessed Feb 2010).
  • 18. Trout AT, Kallmes DF, Gray LA, et al. Evaluation of vertebroplasty with a validated outcome measure: the Roland-Morris Disability Questionnaire. AJNR Am J Neuroradiol 2005; 26: 2652-2657.
  • 19. Kallmes D, Comstock B, Gray L, et al. Baseline pain and disability in the Investigational Vertebroplasty Efficacy and Safety Trial. AJNR Am J Neuroradiol 2009; 30: 1203-1205.
  • 20. Bono CM, Heggeness M, Mick C, et al. North American Spine Society newly released vertebroplasty randomized controlled trials: a tale of two trials. Spine J 2009; Oct 10. [Epub ahead of print].
  • 21. Hancock M, Herbert R, Maher C. A guide to interpretation of studies investigating subgroups of responders to physical therapy interventions. Phys Ther 2009; 89: 698-704.
  • 22. Rousing R, Andersen M, Jespersen S, et al. Percutaneous vertebroplasty compared with conservative treatment in patients with painful acute or subacute osteoporotic vertebral fractures: three-months follow-up in a clinical randomized study. Spine 2009; 34: 1349-1354.
  • 23. Klazen CA, Verhaar HJ, Lampmann LE, et al. VERTOS II: Percutaneous vertebroplasty versus conservative therapy in patients with painful osteoporotic vertebral compression fractures; rationale, objectives and design of a multicenter randomized controlled trial. Trials 2007; 8: 33.

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Responses are now closed for this article.