The Western Australian Audit of Surgical Mortality: outcomes from the first 10 years

The Western Australian Audit of Surgical Mortality (WAASM) is an external, independent, peer-reviewed audit of deaths of patients in hospital while under the care of a surgeon. A pilot based on the Scottish Audit of Surgical Mortality commenced in 2001,1 and the program expanded to the whole state in 2002. The Royal Australasian College of Surgeons (RACS) established the Australian and New Zealand Audit of Surgical Mortality (ANZASM) in 2005.2

Here, we review WAASM from 1 January 2002 to 31 December 2011.

Detailed methods are available in the WAASM annual reports.3 In brief, hospitals notify WAASM of deaths of patients that occur while under the care of a surgeon. Notification is independent of the surgeon. WAASM sends the treating surgeon a semistructured proforma, which is completed, returned and scrutinised by a first-line assessor.

About 10% of cases undergo a detailed second assessment. Case notes are sent to a second assessor, who is normally a consultant in the same specialty working in a different hospital. The second assessment is confidential but not blind, as the assessor knows the identity of the surgeon. The second assessor sends a report to WAASM, who forwards this to the surgeon. The identity of the external reviewer is never revealed to the treating surgeon.

The primary purpose of WAASM is educational. Feedback is provided to the surgeon through individual case reports, and to all surgeons in booklets of de-identified case-note reviews, newsletters, annual reports and multidisciplinary symposia.

WAASM is protected under the Commonwealth Qualified Privilege Scheme.4

In the first 10 years of WAASM, the annual number of deaths peaked at 740 in 2006, before falling 22.0% to 577 in 2011 (Box). During the audit period, the population of Western Australia increased 22% from 1.9 to 2.3 million.5 Surgical deaths per 100 000 population fell 31.7% from a peak of 35.9 in 2006 to 24.6 in 2011 (an overall fall of 29.6% from 2002 to 2011). The regression trend in deaths per 100 000 population was fitted with the R statistical language using a restricted cubic spline with five knots. Both the regression (P = 0.002) and its non-linear component (P = 0.001) were statistically significant. Deaths per 100 000 surgical separations fell from 35.9 in 2006 to 24.6 in 2011.

Following are some representative examples that illustrate how WAASM influenced care.

During its first 10 years, WAASM recorded a 30% reduction in the number of deaths per 100 000 population. The pattern observed — no initial change while data are collected, analysed and disseminated, followed by improved outcomes — is in keeping with numerous surgical audits.19-21 Recent examples of audits incorporating external peer review and risk-adjusted data are the American College of Surgeons National Surgical Quality Improvement Program, which recorded decreases in both mortality and morbidity during its first 10 years,22 and the UK’s National Adult Cardiac Surgery Audit that showed a 25% reduction in mortality, with outcomes now superior to the European average.23

The observed improvements are likely to be attributable to a combination of influences. First, as described, some changes are undoubtedly a direct consequence of WAASM, and strong circumstantial evidence suggests that WAASM has had an even wider influence. For example, when WAASM commenced, WA had few beds in high-dependency units. WAASM repeatedly noted that a failure to use a high-dependency unit may have contributed to some deaths and supported a multidisciplinary symposium on this topic. Many hospitals have since opened high-dependency units or now accept patients into intensive care units rather than “specialling” them on wards.

Second, it is highly likely that the Hawthorne effect — where those under observation change their practice — contributed to these changes.24 For example, as a consequence of restricting pancreatectomies to teaching hospitals, most upper gastrointestinal cancer resections now occur in the same institutions. It is also highly likely that some patients avoided secondary complications, and thus death, because of increased consultant presence during returns to theatre.

Third, there is substantial anecdotal evidence that surgeons in WA are increasingly admitting high-risk (especially emergency) patients into teaching hospitals, in recognition of their higher risk of complications. During this audit period, the number of patients dying in major private hospitals more than halved, while the proportion of deaths occurring in teaching hospitals increased from 74% to 84% (13.5%). This redirection may be entirely appropriate but has obvious implications if the public sector is required to manage an increasing number of older, frail and emergency patients.

These observations are in keeping with reports suggesting that any difference in outcome between high- and low-volume surgeons and hospitals may not be related to the initial complication rate, but to the management of complications — so-called failure to rescue.25-28 These studies suggest that initial complications are usually related to patient factors, such as advanced age and medical comorbidities, and rates are similar in high- and low-volume hospitals. The ability to rescue patients after an initial complication seems to depend on facilities found in large hospitals, such as high nurse-to-patient ratios, junior staff, high-dependency units, interventional radiologists and cardiologists, and greater consultant availability.28 Failure to rescue may be five times higher in poorly performing hospitals and it is likely this, rather than the initial complication rate, that determines differences in outcome.

The hallmark feature of WAASM is its external, independent, blind peer review. WAASM has found that treating surgeons only record 18% of the deficiencies of care that are identified by external reviewers.29 External review is normal in other safety-critical industries. The case-note review booklets given to surgeons have consistently been identified as providing the most valued feedback.

Although all WA surgeons now participate in WAASM, this has taken 10 years to achieve. In 2002, many surgeons expressed reservations about personal and medicolegal consequences of participation. Without qualified privilege, WAASM would never have commenced. Although perceived medicolegal concerns have not been borne out, some surgeons remain resistant to participation or complete the proformas in a less than satisfactory manner. This has been an issue in other audits.30 The RACS has recognised that non-participation is no longer defensible. Since 2010, participation in ANZASM has been a mandatory part of its Continuous Professional Development Program, and the proportion of Fellows verified has increased. A current certificate is essential to register with the Medical Board of Australia.

Hospitals and health departments need to accept responsibility when audits demonstrate a need for service redesign. Although there is overwhelming evidence to support the separation of emergency and elective surgical patients, there has been some reluctance to undertake the fundamental reorganisation required.31

A major difference between Australia and the UK, where consultant compliance with such audits is reviewed annually, is that the latter has matched its rhetoric with contractual muscle and funding.

In conclusion, we found that a surgical mortality audit can change practice and improve outcomes. However, surgical audit should be extended beyond mortality. While it will not be possible to audit every aspect of surgical care, targeting key events, such as unplanned returns to theatre, could identify most significant complications. Governments, hospitals and the RACS must each make it clear that full and complete participation in clinical audits is the only acceptable standard. WAASM has provided unequivocal evidence that some form of external review is essential.

A natural progression is open publication of outcome data. Those who pay for health care (governments) and those who use it (the public) want to know what they are purchasing and using. The profession should rise to the challenge of agreeing to greater transparency.32 Identifying regions in ANZASM reports would be an appropriate start.

Deaths notified to the Western Australian Audit of Surgical Mortality, 2002–2011