Public reporting of clinician-level data

There is much debate about public disclosure of individual doctors’ performance to increase hospital quality and safety, but research is lacking

In 2016, media coverage of a cluster of preventable deaths of babies born at a Victorian health service1 shone a spotlight in Australia on the role of public reporting of hospital performance data in assuring quality and safety. The subsequent Victorian government review2 suggested that the Victorian health system must develop a culture of candour with improved transparency at every level of the hospital system “through greater public reporting of outcomes data and support for a just culture in hospitals”.2 Other failures in hospital quality have similarly triggered inquiries and health system reform in Australia.3,4 For example, Queensland’s Bundaberg Hospital scandal in 2005 triggered changes to public reporting to further encourage cultures in hospitals to move away from the “name–shame–blame” approach.5

Public reporting of hospital performance data is considered to be a mechanism to improve transparency and accountability, drive quality improvement, and inform health care decision making.6,7 It is an expanding phenomenon in Australia and elsewhere,8 with the most explosive growth occurring in the United States, where there are reportedly over 4000 public reporting sites with more in development.9 Yet the potential pitfalls and opportunities related to public reporting have long been debated.3,4

Information about health care structures, costs, processes, outcomes and patient experiences is reported publicly, to various levels of aggregation and granularity, including at hospital, operational unit within a hospital, and identifiable, individual clinician level.8 Evidence on the impact of public reporting tends to indicate that it can have a positive impact on quality improvement in hospitals, partly through desire to maintain reputation (individual, institutional, political) and avoid unfavourable media publicity.6,7,10-13 Its impact on consumer decision making is less clear.7,14

Unlike in Australia, public reporting in the United Kingdom includes ratings of individual hospital specialists’ performance.8 Some websites in the US also report identifiable data on individual clinicians, although it is mostly voluntary; a well known example is the public reporting of cardiac surgery performance.4,6 Public reporting at the individual clinician level is controversial, owing to the methodological issues related to risk adjustment and potential for unintended negative consequences.8,15,16

Australian context

Australia has been described as lagging behind and being “less advanced” than many countries when it comes to public reporting.5,17 In 2011, the Australian government introduced the MyHospitals website, the only nationally consistent and comparable public reporting system for public and private providers (mandatory for public hospital reporting but voluntary for private sector providers). Currently, owing to methodological challenges and lack of data, just seven of 17 indicators are reported. Reporting of hospital standardised mortality ratios is planned for MyHospitals despite contention about its validity and reliability as an indicator of quality. Although hospital standardised mortality ratios are calculated in a number of countries, their routine public reporting is not widespread.15 On the other hand, public reporting of patient experience, also planned for MyHospitals, is becoming more common across the world — although identifiable reporting of this to the individual clinician level is not widespread.8

In addition to public reporting, there are other mechanisms for quality improvement related to clinician-level data that can also affect hospital quality improvement. For example, confidential, non-public auditing and reporting of surgical mortality via the Royal Australasian College of Surgeons occurs in most Australian states and territories. The audit involves providing independently reviewed feedback to individual surgeons and de-identified case reports to all surgeons. A study of the first ten years of the audit in Western Australia attributed a significant reduction in the number of surgery-related deaths to changes in practice directly related to the audit.18 Use of internal feedback and reports is thought to increase clinicians’ trust but perhaps lower public trust.3

The ethical obligation of disclosing clinician-level data has been widely explored.4,6,12,15,19 In theory, information about an individual clinician’s performance is as relevant to informed consent as disclosure of rare clinical risks routinely discussed with patients.6,12,19 Ethical arguments commonly made in support of public disclosure of clinician-level data include that it can increase patient autonomy and ability to make better informed decisions about treatment options (related to informed consent), and that such disclosure can help clinicians meet their professional accountability obligations.4,6,12 However, the ethics of making meaning from poor quality or inadequately interpreted data, in addition to the high cost of collecting and meaningfully analysing quality, comparable data, are perhaps contributory reasons why Australia has been particularly cautious in its roll-out of its national system of public reporting of hospital performance.

International perspectives

The reporting of clinician (specifically surgeon)-level data (eg, report cards) has been described as an “emerging opportunity to improve care and inform patients”,9 and reportedly met with interest and enthusiasm in some quarters15 but with caution in others.3,4,16 Arguments for publicly reporting clinician-level data are often based on the ethical arguments previously outlined; that is, to facilitate patients’ informed choice and to transparently demonstrate safety.6 Overall, however, there remains little evidence that health care consumers are aware of or use such information, that its reporting has stimulated quality improvement activities or improved clinician outcomes.7 In the US, a survey of 185 surgeons found that that a little over half of respondents (53%) opposed individual surgeon-level public reporting and that just 18% indicated that their practice would change in response to such reporting.20 There is some evidence that individual clinician-level reporting may be an incentive for low-performing surgeons to improve.9

The arguments against public reporting of clinician-level data are many and widespread — although often based on perceptions rather than on research.13,15-17,20 In England, negative consequences of clinician and hospital rating systems have been associated with reduced staff morale, erosion of public trust, the deliberate manipulation of data, altered behaviours to obtain strategic advantage (gaming), bullying to maintain star ratings, and incorrect inferences being made about performance data.16 Studies from the US have suggested that identifiable clinician-level public reporting may lead some clinicians to avoid treating high risk patients, a practice that weights those surgeons’ outcome statistics more favourably through selection of less risky patients.13,15 Gaming has also been observed in the US.9 Further, concern has been raised that robust and reliable data are often lacking on which to base performance reporting (including lack of appropriate risk adjustment), that consumers are not necessarily able to appropriately interpret the data, and that some good clinicians could potentially suffer reputational harm as a result.6,9,14,15

It has been proposed that clinician-level reporting needs to be made more meaningful for its benefits to be realised. To this end, work is underway in the US to develop a tool for use by health care consumers that takes into account individual risk factors and demographic information. This will enable individuals to assess their perioperative risks at different hospitals.9

Despite debate, there has been very little research on perceived and actual impacts and consequences of public reporting of clinician-level performance data. Methodologically comparable research, controlled studies, and studies of accrued effects of public reporting are particularly lacking.13

Summary

Individual clinician-level public reporting is not currently part of any official avenue of public reporting of hospital performance data in Australia. Local and international research on the impacts of clinician-level reporting is, as yet, limited in its conclusions. Given the potential interest of consumers and the impact that such reporting may have on clinicians, it is an issue that merits further research and debate. If public reporting of clinician-level data were to become a reality in Australia, the experiences of other countries should be carefully considered with particular attention to lessons learned.3,16,20,21 Public reporting of hospital performance data at any level, however, remains just one mechanism for the continuous improvement of hospital quality and safety.