Editorials
Health services administration
Volume 194 - Issue 12

Asking the hard questions about safety and quality indicators

Author:  David I Ben-Tovim

Med J Aust 2011; 194 (12): 623-624. || doi: 10.5694/j.1326-5377.2011.tb03143.x
Published online: 20 June 2011
Download PDF (1 MB)
Share: Share on Blue Sky Share on Facebook Share on LinkedIn Share by email

We need to balance the technical challenges of hospital standardised mortality ratios with the need to improve care processes

That “sunshine is the best disinfectant” is attributed to the distinguished American jurist Louis Brandeis, who spent much of his career supporting individual rights in the context of corporate and monopoly power.1 State and federal governments, in conjunction with bodies such as the Australian Commission on Safety and Quality in Health Care and the Australian Institute of Health and Welfare, are developing indicators for the safety and quality of Australian hospital care, presumably with the intention of using them to illuminate some of the inner workings of our hospital system for the benefit of both health care providers and the community at large. The challenges posed by such a program are illustrated in this issue of the Journal by Scott and colleagues2 and Gallagher and Krumholz.3

Health care professionals face a basic dilemma. The vast majority of us are hard working, conscientious and altruistic. Yet the hospitals in which we work collectively expose patients to substantial risks over and above those posed by their clinical conditions, and improving hospital safety and quality is work that ultimately can only be done by hospital staff. The United Kingdom Department of Health recently summed up the task of a safety and quality indicator such as the hospital standardised mortality ratio (HSMR):

A high HSMR is a trigger to ask hard questions. Good hospitals monitor their HSMRs actively and seek to understand where performance may be falling short and action should not stop until the clinical leaders and the Board at the hospital are satisfied that the issues have been effectively dealt with.4

The challenge is that exposure to a safety and quality indicator will trigger hard questions about the technical qualities of the indicator rather than underlying care processes, and opportunities for improvement will be lost.

The technical issues canvassed2,3 include the value of coded administrative data as a source for risk adjustment; analysis and interpretation of data from small hospitals; and problems related to classifying and coding hospital palliative care provision. All require detailed work, such as is in progress for HSMRs.5 Furthermore, the Australian Commission on Safety and Quality in Health Care is going through a rigorous development process for at least 16 measures.6 A dilemma in indicator development is whether to accept the merely good or await the perfect. Discussion about technical matters should not divert attention from the basic questions about what available indicators such as relative hospital mortality actually measure, and how they, and other indicators, should be used and distributed.

Mortality indicators compare observed mortality against a risk-adjusted expected mortality derived from a broader reference source. Differences are silent as to cause. They are simply a prompt for hard questioning. Process measures of adherence to evidence-based care pathways are well established quality indicators that provide immediate feedback on areas for improvement. Mortality measures do not correlate well with process measures.2 Does this undermine the validity of mortality as a safety and quality indicator? Only if process measures are taken as the gold standard of hospital safety and quality. But what if a measure of getting evidence-based steps right (a process measure) is not directly related to a measure of what happens when things go wrong? What if hospital mortality measures are at a tangent to process measures, rather than directly consequential?7 Then the hard questions provoked by an elevated mortality rate of any kind might do well to begin with examining the capacity to rescue patients when things go wrong,8 as they will in even the best-organised departments.

Scott and colleagues2 provide excellent practical advice on how institutions might use indicators of various kinds to provoke hard questions. They are wary, however, of providing hospital staff, or the public at large, with comparative information on hospital outcomes. The efficacy of comparative and public reporting as a prompt to improving hospital safety and quality cannot be settled by rigorous scientific means. It is just not possible to carry out randomised double-blind trials of accurate versus inaccurate dummy comparative reporting. The extent of public reporting is a public policy issue in which the risks to institutional reputation and morale have to be balanced against a need for accountability and the opportunity for informed choice by current and future users of health care institutions.9 Gallagher and Krumholz note that Australia has lagged behind other countries in publicly reporting hospital outcomes.3 As Brandeis might have argued, one of the most important virtues of public reporting is that it makes it harder for vested interests, whether they are in government or institutions, to suppress unwelcome information. No doubt the debate has just begun.

Author

  • David I Ben-Tovim PhD, MB BS, FRANZCP1,2
  • 1. Redesigning Care and Clinical Epidemiology Units, Flinders Medical Centre, Adelaide, SA.
  • 2. Faculty of Health Sciences, Flinders University, Adelaide, SA.

Correspondence: david.ben-tovim@health.sa.gov.au

Competing interests

The Clinical Epidemiology Unit, Flinders Medical Centre has received grant support from the Australian Commission on Safety and Quality in Health Care for analysis of hospital mortality data.

References

Provenance: Commissioned; externally peer reviewed.

View this article on Wiley Online Library