Time to mandate data release and independent audits for all clinical trials

Editorials and commentaries in some high-profile journals herald an upcoming revolution in personalised oncology.1 However, any new treatment can only be considered an advance if it:

• extends the life of the patient;

• improves quality of life;

• reduces the toxicity of the current best treatment; and/or

• reduces costs.

Definitive proof of therapeutic benefit relies on freely accessible, high-quality data and their independent evaluation. Unfortunately, open access to de-identified patient data and statistical analyses remains unavailable, so only limited verification of claims emanating from commercially sponsored clinical trials is possible. This restriction reinforces concerns about reporting of trials in general, as “overestimation of the clinical benefit of a drug” is well documented.2

Further, reliance on progression-free survival (PFS) as a surrogate for the clinical benefit of a drug is a risky undertaking. PFS is subjective, as it is based on interpretation of radiological tumour size, whereas overall survival (OS) is objective and unambiguous. Despite the inherent subjectivity of PFS, Genentech requested its use as a basis for the approval of bevacizumab (Avastin) for first-line treatment of locally recurrent or metastatic human epidermal growth factor receptor 2 (HER2)-negative breast cancer.3 However, the usefulness of PFS as a surrogate for OS, therapeutic benefit and accelerated drug approval is controversial.4-6 To understand why, it is prudent to carefully re-examine the original data, particularly as time-constrained clinicians may be unfamiliar with important details of bevacizumab’s accelerated approval.

In 2007, the United States Food and Drug Administration (FDA) Oncologic Drugs Advisory Committee evaluated data from a report of the E2100 trial,7 in which Genentech claimed an impressive 5.5-month increase in median PFS (mPFS) as a therapeutic benefit, but showed no improvement in median OS (mOS).3 Hence, the participating patients did not live longer with bevacizumab treatment. As no correlation existed between mPFS and mOS or quality of life, we asked: “What, then, are the benefits of this treatment?”8

The uncertainties regarding the strengths of Genentech’s claims were exposed by the Committee’s analysis,3 which listed many “significant protocol deviations”. These deviations were tabulated in the Committee’s analysis and included: stratification errors and treatment beyond progression (Table 3); absent radiographs for some participants (Table 5); discordance between the independent review facility and the trial investigators in PFS determination, with incorrect dates for disease progression, including a massive discordance rate of 51% of PFS date (Table 8); and more frequent dose modifications, omissions, delays and reductions in the bevacizumab arm (Tables 10 and 11).3 The Committee disagreed with Genentech’s cause-of-death attribution in several instances (Tables 15, 16 and 17) and documented a 20% increase in the incidence of grade 3–5 adverse events (including hypertension and neutropenia) in the bevacizumab arm (Tables 13 and 14).3

The Committee also analysed a precursor randomised phase III trial from Genentech (denoted AVF2119g),9 which compared bevacizumab plus capecitabine with capecitabine alone in patients with previously treated metastatic breast cancer.3 The increase in mPFS in AVF2119g was a non-significant 3 weeks, in striking contrast to the large 5.5 month value in the E2100 trial. The Committee took cognisance of the serious adverse events (Tables 19 and 20) and concluded that this trial “failed to demonstrate a statistically significant effect on PFS and overall survival”.3

Given these data, the Committee voted against approval of bevacizumab for first-line treatment of locally recurrent or metastatic HER2-negative breast cancer.

Despite this recommendation, which was based on independent scientific, clinical and biostatistical analyses, bevacizumab received accelerated approval with the proviso that further confirmatory trials be conducted.4,5

Three years later, the confirmatory trials, AVADO10 and RIBBON-1 (Regimens in Bevacizumab for Breast Oncology),11 were completed. Bevacizumab plus docetaxel was compared with docetaxel plus placebo in AVADO, and capecitabine, anthracyclines or taxanes plus either bevacizumab or placebo were compared in RIBBON-1. The previous stunning 5.5-month improvement in mPFS was not seen. AVADO and RIBBON-1 yielded mPFS values of 0.8, 1.2, 1.9 and 2.9 months — again, with no improvement in mOS. Patients did not live longer and both trials confirmed “the serious risks associated with bevacizumab”.4 With this new evidence, the FDA initiated proceedings to withdraw approval for bevacizumab for metastatic breast cancer,5 a move endorsed in editorials in the Journal of Clinical Oncology and Nature Biotechnology, which concluded, respectively, that “the outcomes were arguably not clinically compelling”12 and that “if lack of [drug] efficacy in the face of toxicity is insufficient to reverse an accelerated approval, then what is?”.13

It is illuminating to compare the mPFS values from the four bevacizumab breast cancer trials (0.7, 0.8, 1.2 1.9, 2.9 and 5.5 months),7,9-11 with values from the randomised phase III trials of bevacizumab in prostate, ovarian, gastric, pancreatic and colorectal cancer (0.4, 0.6, 0.9, 0.9, 1.0, 1.4, 1.4, 1.7, 2.4, 3.8 and 4.4 months).6 First, the 5.5-month value on which bevacizumab received accelerated approval is the extreme outlier. Second, statistically significant increases in mOS occurred in only two of the above 17 patient sets.6 Clearly, statistically significant increases in mPFS were not reflected in mOS.

Thus, irrespective of whether tumour size is increasing, decreasing, or remaining stable under drug treatment, tumour size changes are extremely poor predictors of how long a patient will live. The above data show that PFS is not a surrogate for OS.

Evaluating the therapeutic benefit of other anti-cancer drugs requires similar in-depth data analyses. In the case of cetuximab for first-line treatment of metastatic colorectal cancer and the use of KRAS mutations as biomarkers in tumour samples, the increase in mPFS was only 0.9 months, with no increase in mOS.14 As with bevacizumab, some physicians with no ties to the study concluded that this small difference is “clinically irrelevant”.15

Similarly, claims of therapeutic benefit for rituximab in treatment of chronic lymphocytic leukaemia16 and chemotherapy-sensitive low-grade follicular lymphoma17 have been questioned, particularly as these claims were based on PFS, a largely clinically irrelevant end point in these usually indolent diseases.18-20

In breast cancer, claims for the superior efficacy and safety of anastrozole, an expensive, often toxic aromatase inhibitor, evaluated in postmenopausal women with early-stage breast cancer,21 have been challenged.22 The data failed to show a survival advantage over tamoxifen, which is cheaper and well tolerated.21

We further contend that claims of therapeutic benefit based on PFS — from the recent trials of sunitinib and everolimus in low-grade and indolent pancreatic neuroendocrine tumours,23,24 zalutumumab in recurrent or metastatic squamous cell carcinoma of the head and neck25 and vandetanib in advanced non-small cell lung cancer26 — all require additional trials before they can be considered therapeutically robust.

Most of the above drugs, all with questionable therapeutic benefits, are very expensive. For example, approximate costs per month for an average patient for bevacizumab, everolimus, sunitinib or cetuximab are AUD $3400, $5700, $5800 and $7000, respectively.27 Some newer drugs, recently approved in the US and already in use in trials in Australia, are even costlier (ipilimumab for metastatic melanoma sells at US$120 000 wholesale for a four-dose course of treatment given over 3 months).28

In summary, many drugs will add a significant burden to the Australian health care system, and hence all claims based on PFS by authors of pharmaceutical-company- or academic-sponsored trials need to be carefully scrutinised by independent experts before regulatory approval.

How can the evaluation of therapeutic benefit be improved?

A pragmatic example has been set by the molecular, neurobiological and physical sciences communities. First, all de-identified raw data should be lodged in approved, publicly accessible databases where the data conform to minimum information standards and are in a form suitable for independent statistical scrutiny, as exemplified by the US National Center for Biotechnology Information Gene Expression Omnibus (http://www.ncbi.nlm.nih.gov/geo). Second, an independent evaluation of the data conducted by professionals with no ties to, or financial compensation from, the sponsor or its surrogates should accompany the published abstract in medical journals. This “accompanying abstract” constitutes an independent clinical audit. Public companies cannot audit their own financial returns, and it is even more important that companies whose activities involve billions of public dollars in health care expenditure should abide by standards of transparency that can be independently verified using the highest standards of scientific excellence.

As a recent MJA commentary stated:

Medical journals and their editors have a choice — to be viewed as “an extension of the marketing arm of pharmaceutical companies”,30 or to be beacons of transparent data processes that inform clinicians, improve patient treatment, and provide high standards on which governments, health care providers and patients can have confidence.

Medical journals should demonstrate strong leadership by mandating open access to detailed clinical trial protocols and de-identified raw study data. They should insist on independent audits of data, concomitant publication of an “accompanying abstract”, and lodgement of the data in independent databases; these three actions should be a precondition for publication.