The Western Australian Audit of Surgical Mortality (WAASM) is an external, peer-reviewed audit of all deaths that occur in hospital of patients under the care of a surgeon.
We conducted a retrospective analysis of prospective audit data collected from 1 January 2002 to 31 December 2011.
The annual number of deaths peaked in 2006, then fell 22% by 2011. After correcting for population growth, the overall reduction from 2002 to 2011 was 30% (regression analysis, P = 0.002).
Some changes in practice, such as with pancreatic surgery, can be directly attributed to WAASM. There is strong evidence to suggest that WAASM improved other aspects of care, such as thromboembolic prophylaxis, consultant supervision and fluid management.
A shift of high-risk patients to teaching hospitals, where there is a greater ability to “rescue” patients after complications, may have been an important factor in improved outcomes.
This external, peer-reviewed mortality audit has changed surgical practice and reduced deaths. The same process should be applied to other sentinel events, and the lessons learned can also be extended to non-surgical specialties.
The Western Australian Audit of Surgical Mortality (WAASM) is an external, independent, peer-reviewed audit of deaths of patients in hospital while under the care of a surgeon. A pilot based on the Scottish Audit of Surgical Mortality commenced in 2001,1 and the program expanded to the whole state in 2002. The Royal Australasian College of Surgeons (RACS) established the Australian and New Zealand Audit of Surgical Mortality (ANZASM) in 2005.2
Detailed methods are available in the WAASM annual reports.3 In brief, hospitals notify WAASM of deaths of patients that occur while under the care of a surgeon. Notification is independent of the surgeon. WAASM sends the treating surgeon a semistructured proforma, which is completed, returned and scrutinised by a first-line assessor.
About 10% of cases undergo a detailed second assessment. Case notes are sent to a second assessor, who is normally a consultant in the same specialty working in a different hospital. The second assessment is confidential but not blind, as the assessor knows the identity of the surgeon. The second assessor sends a report to WAASM, who forwards this to the surgeon. The identity of the external reviewer is never revealed to the treating surgeon.
The primary purpose of WAASM is educational. Feedback is provided to the surgeon through individual case reports, and to all surgeons in booklets of de-identified case-note reviews, newsletters, annual reports and multidisciplinary symposia.
WAASM is protected under the Commonwealth Qualified Privilege Scheme.4
In the first 10 years of WAASM, the annual number of deaths peaked at 740 in 2006, before falling 22.0% to 577 in 2011 (Box). During the audit period, the population of Western Australia increased 22% from 1.9 to 2.3 million.5 Surgical deaths per 100 000 population fell 31.7% from a peak of 35.9 in 2006 to 24.6 in 2011 (an overall fall of 29.6% from 2002 to 2011). The regression trend in deaths per 100 000 population was fitted with the R statistical language using a restricted cubic spline with five knots. Both the regression (P = 0.002) and its non-linear component (P = 0.001) were statistically significant. Deaths per 100 000 surgical separations fell from 35.9 in 2006 to 24.6 in 2011.
In 2004, WAASM reported that a consultant was present for less than half of the operations undertaken in teaching hospitals after which the patient died.3 The teaching hospitals used these data to advocate for the separation of emergency and elective general surgery. In 2006, the WA Department of Health agreed to separate emergency and elective general surgery in teaching hospitals, and established an “on-call day surgeon” with access to a daytime emergency theatre. Between 2002 and 2011, the number of deaths after emergency surgery fell by 22% overall and by 38% for patients aged 65 years or older. Data from Perth underpinned the RACS position paper on the separation of emergency and elective surgery.6
The United Kingdom’s National Confidential Enquiry into Patient Outcome and Death and the Scottish Audit of Surgical Mortality have both recently emphasised the importance of early consultant input in managing emergency patients.7,8
In early reports, WAASM noted that teaching hospital consultants directly performed or assisted in only 40% of first returns to theatre and 35% of second returns. By 2012, these rates of consultant supervision had risen to 51% and 57%, respectively.9 In 2009, ANZASM found consultant participation in WA was greater than in other states.10
An early observation was the failure to appropriately administer deep vein thrombosis prophylaxis. WAASM hosted a symposium on this issue in 2003 and continues to highlight such failures. Subsequent data have demonstrated a fall in the number of deaths secondary to a pulmonary embolus. WAASM concerns preceded publication of the Australian guidelines on prevention of venous thromboembolism.11
An increasing number of patients take anticoagulant drugs, such as antiplatelet agents after coronary stent insertion and warfarin for atrial fibrillation or mechanical heart valves. In 2005, after noting deaths occurring secondary to suboptimal perioperative anticoagulant management, WAASM organised a multidisciplinary symposium on this topic. Since that symposium, there has not been a death secondary to anticoagulation mismanagement.
Poor postoperative fluid management is an international problem. In 2008, WAASM hosted a symposium that included a Scottish anaesthetist who had been involved in the production of the Scottish Intercollegiate Guidelines on postoperative management.12 Subsequently, there has been a fall in the number of deaths associated with poor fluid management (52 in 2007, nine in 2011). Nationally, poor fluid management remains a problem.13
WAASM noted that death after pancreaticoduodenectomy (Whipple resection) was occurring more frequently than expected, and a statewide analysis confirmed a postoperative mortality rate of 16%, compared with an expected rate of around 2%.14 A working party chaired by WA’s Chief Medical Officer determined that major pancreatic resections should only be undertaken in teaching hospitals with the appropriate personnel and multidisciplinary support. In the past 2 years (May 2010 to May 2012), there have been no deaths after a Whipple resection.
This is an area of current national and international concern.15 WAASM has repeatedly reported that delay in recognising deterioration in a patient’s condition is the leading cause of death. In 2011, WAASM hosted a multidisciplinary symposium on clinical deterioration, with the most important feedback being that nurses better understood the detrimental impact of delay and the importance of escalating care early.16
In 2005, the WA Department of Health introduced the Western Australian Review of Mortality.17 This extended the concepts of WAASM to all specialties although, unlike WAASM, there is no routine external peer review.
WAASM surveyed surgeons in 200318 and 2012 (unpublished). In both surveys, surgeons acknowledged WAASM had influenced their management of patients. The 2012 survey showed the greatest influence had occurred in targeted areas, such as consultant supervision for returns to theatre.
During its first 10 years, WAASM recorded a 30% reduction in the number of deaths per 100 000 population. The pattern observed — no initial change while data are collected, analysed and disseminated, followed by improved outcomes — is in keeping with numerous surgical audits.19-21 Recent examples of audits incorporating external peer review and risk-adjusted data are the American College of Surgeons National Surgical Quality Improvement Program, which recorded decreases in both mortality and morbidity during its first 10 years,22 and the UK’s National Adult Cardiac Surgery Audit that showed a 25% reduction in mortality, with outcomes now superior to the European average.23
The observed improvements are likely to be attributable to a combination of influences. First, as described, some changes are undoubtedly a direct consequence of WAASM, and strong circumstantial evidence suggests that WAASM has had an even wider influence. For example, when WAASM commenced, WA had few beds in high-dependency units. WAASM repeatedly noted that a failure to use a high-dependency unit may have contributed to some deaths and supported a multidisciplinary symposium on this topic. Many hospitals have since opened high-dependency units or now accept patients into intensive care units rather than “specialling” them on wards.
Second, it is highly likely that the Hawthorne effect — where those under observation change their practice — contributed to these changes.24 For example, as a consequence of restricting pancreatectomies to teaching hospitals, most upper gastrointestinal cancer resections now occur in the same institutions. It is also highly likely that some patients avoided secondary complications, and thus death, because of increased consultant presence during returns to theatre.
Third, there is substantial anecdotal evidence that surgeons in WA are increasingly admitting high-risk (especially emergency) patients into teaching hospitals, in recognition of their higher risk of complications. During this audit period, the number of patients dying in major private hospitals more than halved, while the proportion of deaths occurring in teaching hospitals increased from 74% to 84% (13.5%). This redirection may be entirely appropriate but has obvious implications if the public sector is required to manage an increasing number of older, frail and emergency patients.
These observations are in keeping with reports suggesting that any difference in outcome between high- and low-volume surgeons and hospitals may not be related to the initial complication rate, but to the management of complications — so-called failure to rescue.25-28 These studies suggest that initial complications are usually related to patient factors, such as advanced age and medical comorbidities, and rates are similar in high- and low-volume hospitals. The ability to rescue patients after an initial complication seems to depend on facilities found in large hospitals, such as high nurse-to-patient ratios, junior staff, high-dependency units, interventional radiologists and cardiologists, and greater consultant availability.28 Failure to rescue may be five times higher in poorly performing hospitals and it is likely this, rather than the initial complication rate, that determines differences in outcome.
The hallmark feature of WAASM is its external, independent, blind peer review. WAASM has found that treating surgeons only record 18% of the deficiencies of care that are identified by external reviewers.29 External review is normal in other safety-critical industries. The case-note review booklets given to surgeons have consistently been identified as providing the most valued feedback.
Although all WA surgeons now participate in WAASM, this has taken 10 years to achieve. In 2002, many surgeons expressed reservations about personal and medicolegal consequences of participation. Without qualified privilege, WAASM would never have commenced. Although perceived medicolegal concerns have not been borne out, some surgeons remain resistant to participation or complete the proformas in a less than satisfactory manner. This has been an issue in other audits.30 The RACS has recognised that non-participation is no longer defensible. Since 2010, participation in ANZASM has been a mandatory part of its Continuous Professional Development Program, and the proportion of Fellows verified has increased. A current certificate is essential to register with the Medical Board of Australia.
Hospitals and health departments need to accept responsibility when audits demonstrate a need for service redesign. Although there is overwhelming evidence to support the separation of emergency and elective surgical patients, there has been some reluctance to undertake the fundamental reorganisation required.31
In conclusion, we found that a surgical mortality audit can change practice and improve outcomes. However, surgical audit should be extended beyond mortality. While it will not be possible to audit every aspect of surgical care, targeting key events, such as unplanned returns to theatre, could identify most significant complications. Governments, hospitals and the RACS must each make it clear that full and complete participation in clinical audits is the only acceptable standard. WAASM has provided unequivocal evidence that some form of external review is essential.
A natural progression is open publication of outcome data. Those who pay for health care (governments) and those who use it (the public) want to know what they are purchasing and using. The profession should rise to the challenge of agreeing to greater transparency.32 Identifying regions in ANZASM reports would be an appropriate start.