To the Editor: In their recent article on the use of modelling in pharmacoeconomics to estimate the potential benefits, risks and costs of preventive drugs, Liew and colleagues highlighted important strengths and limitations of this technique.1 One limitation is that modelling is discretionary: different analysts elect different models and get different answers. We argue that modelling is the first step. The next step is testing the predictive validity of the model by systematically collecting cost and effectiveness data over a period of time. Then the predictions of the original analysis could be compared to what actually transpired.

The goal of pharmacoeconomics is the most accurate estimation of costs and benefits. The more these variables are truly study outcomes (that is, experimental and not constrained by assumptions of the economic model), the more valid the process. This will help reduce model discretion, improve data quality and increase the likelihood that the experiment could be replicated independently, the acid test of validity. In reducing the discretion inherent in pharmacoeconomics, we can allay the concerns of those who have questioned its underlying theory2 and validity.3

If one does not know the benefit of a drug, one conducts a study. Equally, if one does not know the cost of a drug, one needs a study. This has been the impetus for randomisation in design of pharmacoeconomic studies, thereby decreasing reliance on economic modelling.4 Furthermore, there are well-tested methods for quantifying the uncertainty of estimates obtained with randomised studies. In contrast, economic modelling assesses the robustness of the model assumptions, as reflected in the estimate, using sensitivity analysis. However, this analysis cannot separate uncertainties attributable to the model assumptions, uncertainty inherent in the data put into the model, and uncertainty of outcome estimates. One is left with nostalgia for the simplicity of the null hypothesis.

How can we move forward?

To reduce reliance on modelling and to collect better data, we propose that Australia, with its "culture of evaluation",5 again take the lead by creating a new "conditional listing" category on the Pharmaceutical Benefits Scheme for all drugs, not just preventive therapy. This would be available for selected products with strong biological rationale but inadequate current evidence on cost-effectiveness. By necessity, these would include only high-volume/low-cost and low-volume/high-cost products, as high-volume/high-cost products are rarely developed, and low-volume/low-cost products are not problematic. The sellers would then collect prospective data to substantiate cost-effectiveness of the products or would have them delisted. This would be truly innovative and, like the Pharmaceutical Benefits Advisory Committee itself, a first for the Commonwealth. Certainly, no other jurisdiction is even considering this, let alone proposing systematic study.

With better data, we would learn which models and model assumptions yield accurate predictions. Of course, many challenges and difficulties will need to be addressed regarding this proposal, but in our opinion none are insurmountable, and the benefits of a program of this type clearly exceed the risks.

Competing interests: K R J was a medical regulator for the United States Food and Drug Administration from 1985 to 2001, and now works for M-TAG, a private company that performs clinical, epidemiological and health economic evaluations of drugs, devices and technology.