eMJA     The Medical Journal of Australia

Home | Issues | eMJA shop | My account | Classifieds | Contact | More... | Topics | Search   

Editorials

Public reporting of comparative information about quality of healthcare

Martin N Marshall and Robert H Brook
MJA 2002; 176 (5): 205-206

A greater degree of public reporting of information about healthcare quality is an inevitable and desirable way forward

The Australian Council for Safety and Quality in Health Care (ACSQHC) plans to publish data about the performance of the Australian healthcare system. It is probably inevitable that this kind of information, which is actively disseminated and reported in such a way as to encourage readers to draw comparisons, will be used in the near future by the media, the public and politicians to make public judgements about the relative performance of individual hospitals or even individual doctors or groups of doctors. Initiatives such as these will therefore be perceived as a threat by some health professionals and some organisations. Would this negative response be justified? What might be gained from public disclosure and how can the policy be implemented successfully?

We believe that a negative response to public disclosure in Australia would be counterproductive. Greater openness in healthcare is inevitable. Information is freely available about most areas of modern life and many believe that healthcare is one of the last bastions of protectionism. When millions of dollars are spent on healthcare, those who pay have a right to know that the money is being spent effectively, and the publication of comparative data sends a strong message about the willingness of health professionals and organisations to be accountable.

In addition, public disclosure appears to be an effective way of improving quality.1 There is a growing body of evidence that the current level of quality of care is unacceptable2,3 and that quality-improvement initiatives using confidential data have been largely ineffective at changing the behaviour of health professionals.4 When comparative data are released to the public, it appears to remind providers of the issues and refocuses them towards taking action.5

Arguments in support of the status quo — that the data are inadequate, the public won't understand them and the media will misuse them — are not sustainable if public disclosure is introduced properly. There are lessons that can be learnt from other countries to guide the process of disclosure in Australia. The United States has nearly 15 years' experience of publishing data in the form of "report cards", or "provider profiles". The initiative was launched by the federal government and the momentum has been maintained by a variety of public, private, commercial and not-for-profit organisations. Consumers and purchasers of healthcare were expected to play a key role by selecting high-performing providers, but recent evidence suggests that the providers themselves make greater use of the data than the service users.6

There are some notable examples of improvements in both the processes and outcomes of care associated with the publication of performance data.1 Public reporting in Europe is less well established than in the United States, but hospital "league tables" have been published in the Netherlands for several years, and the UK government plans to introduce incentives linked to a range of publicly reported performance criteria.7

What can we learn from the initiatives that have already been introduced?

  • First, a backlash from some doctors, professional groups and institutions (particularly those seen to be performing badly) is predictable. Some criticisms were justified in the early days of report cards but lessons are being learnt. For example, we know that forcing new initiatives on reluctant professionals is not the most effective way of changing attitudes, and the introduction of report cards is more likely to be successful if doctors are encouraged to take a lead, particularly in selecting the performance measures. Bringing the media on board at an early stage to ensure fair and balanced coverage also helps. In addition, delaying publication for a short period to allow providers time to look at and act upon the data is a useful strategy.

  • Second, it is important that those who publish the data show a commitment to investing in the process and progressively improving the quality of the data and the validity of comparisons arising from the data. However, it makes little sense to "wait for better data" — data will always be imperfect and, as one commentator stated, it is important not to let "perfect be the enemy of good".8 Experience suggests that the process of publication can in itself act as a catalyst for data improvement.

  • Third, the utility of comparative data comes less from making absolute judgements about performance than from the discussion arising from using the data to benchmark performance. There is therefore a strong educational component to the effective use of comparative data, and resources are required to facilitate this process.6

  • Finally, it is important to be cognisant of the risks of publishing comparative data.9 The danger of institutions refusing to treat certain disadvantaged groups in order to improve their apparent performance is well recognised, although probably overstated,10 and can be reduced by careful adjustment of risk and casemix. A tendency to focus on what is being measured at the expense of other areas of practice can be minimised by publishing a wide range of quality indicators. The risk of "short-termism" — an inappropriate focus on annual reporting cycles — can be reduced by ensuring a balance between short-term targets and long-term strategic aims.

A greater degree of public reporting of information about healthcare quality is an inevitable and desirable way forward. Practitioners and policymakers in Australia have an opportunity to ensure that the policy is implemented in a manner that is most likely to produce positive change.

  1. Marshall MN, Shekelle PG, Leatherman S, Brook RH. What do we expect to gain from the public release of performance data? A review of the evidence. JAMA 2000; 283: 1866-1874. <PubMed>
  2. Schuster MA, McGlynn EA, Brook RH. How good is the quality of U.S. health care? Milbank Q 1998; 76: 517-563. <PubMed>
  3. Seddon ME, Marshall MN, Campbell SM, Roland MO. Systematic review of studies of quality of clinical care in general practice in the United Kingdom, Australia and New Zealand. Quality in Health Care 2001; 10: 152-158. <PubMed>
  4. Davis D, Thomson M, Oxman A, Haynes B. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995; 274: 700-705. <PubMed>
  5. Davies H. Public release of performance data and quality improvement: internal responses to external data by US health care providers. Quality in Health Care 2001; 10: 104-110. <PubMed>
  6. Marshall MN, Shekelle PG, Leatherman S, Brook RH. Dying to know: public release of comparative data in health care. London: Nuffield Trust, 2000.
  7. Department of Health. The NHS plan: a plan for investment, a plan for reform. London: Department of Health, 2000.
  8. Hannan EL. Measuring hospital outcomes: don't make perfect the enemy of good! J Health Serv Res Policy 1998; 3: 67-69. <PubMed>
  9. Smith P. On the unintended consequences of publishing performance data in the public sector. Int J Public Adm 1995; 18: 277-310.
  10. Peterson ED, DeLong ER, Jollis JG, et al. The effects of New York's bypass surgery provider profiling on access to care and patient outcomes in the elderly. J Am Coll Cardiol 1998; 32: 993-999. <PubMed>

(Received 10 Sep 2001, accepted 11 Jan 2002)

National Primary Care Research and Development Centre, Williamson Building, University of Manchester, Manchester, UK.

Martin N Marshall, MSc, MD, FRCGP, Professor of General Practice.

RAND Health, Santa Monica, CA.

Robert H Brook, MD, ScD, Director, and Vice President, RAND.

Correspondence: Professor Martin N Marshall, National Primary Care Research and Development Centre, Williamson Building, University of Manchester, Oxford Road, Manchester, UK, M13 9PL. martin.marshallATman.ac.uk

AntiSpam note: To avoid spam, authors' email addresses are written with AT in place of the usual symbol, and we have removed "mail to" links. Replace AT with the correct symbol to get a valid address.

Other articles have cited this article:

Home | Issues | eMJA shop | My account | Classifieds | More... | Contact | Topics | Search

The Medical Journal of Australia    eMJA  

©The Medical Journal of Australia 2002 www.mja.com.au PRINT ISSN: 0025-729X ONLINE ISSN: 1326-5377