eMJA: Wolff et al, Detecting and reducing hospital adverse events: outcomes of the Wimmera clinical risk management program

Research

Detecting and reducing hospital adverse events: outcomes of the Wimmera clinical risk management program

Alan M Wolff, Jo Bourke, Ian A Campbell and David W Leembruggen

MJA 2001; 174: 621-625
For editorial comment, see Barraclough

→ Other articles have cited this article

Abstract - Methods - Results - Discussion - Acknowledgements - References - Authors' details
Register to be notified of new articles by e-mail - Current contents list - More articles on Administration and health services

Abstract

Objectives: To determine if an integrated clinical risk management program that detects adverse patient events in a hospital, analyses their risk and takes action can alter the rate of adverse events.
Design: Longitudinal survey of adverse patient events over eight years of progressive implementation of the risk management program.
Participants and setting: 49 834 inpatients (July 1991 to September 1999) and 20 050 emergency department patients (October 1997 to September 1999) at a rural base hospital in the Wimmera region of Victoria.
Main outcome measures: Rates of adverse events detected by medical record review and clinical incident and general practitioner reporting.
Results: The annual rate of inpatient adverse events decreased between the first and eighth years of the study from 1.35% of all patient discharges (69 events) to 0.74% (49 events) (P < 0.001). Absolute risk reduction was 0.61% (95% CI, 0.23%-0.99%), and relative risk reduction was 44.9% (95% CI, 16.9%-72.9%). The quarterly rate of emergency department adverse events decreased between the first and eighth quarters of monitoring from 3.26% of all attendances (84 events) to 0.48% (12 events) (P < 0.001). Absolute risk reduction was 2.78% (95% CI, 2.04%-3.52%), and relative risk reduction was 85.3% (95% CI, 62.7%-100%).
Conclusions: Adverse patient events can be detected, and their frequency reduced, using multiple detection methods and clinical improvement strategies as part of an integrated clinical risk management program.

Healthcare delivery in hospitals is associated with adverse patient events,^1,2 and clinical risk management aims to reduce the probability of these events. One approach involves detecting adverse events, analysing their causes, estimating their likelihood and consequences and taking appropriate action to prevent the event recurring.

Adverse events can be detected by medical record review³ and clinical incident reporting.⁴ However, after their detection, analysing the events and determining and taking appropriate action to reduce their rates are difficult tasks. Rates have been reduced in other complex industries, such as aviation, by analysing the systems of service delivery in which the events occurred and changing these systems to reduce their probability.⁵ This systems approach contrasts strongly with the blaming of individuals for errors in healthcare.

In this study, we report the effect of an integrated clinical risk management program that used diverse methods to detect adverse patient events in a hospital and a systems approach to their analysis and action to reduce their rates.

Methods

Setting and patients

The study was undertaken at Wimmera Base Hospital in Horsham, 300 km northwest of Melbourne, Victoria. The hospital provides services to 43 000 people in the Wimmera region, including 13 500 in Horsham. Eight specialists and 14 general practitioners live in the town. With the assistance of eight hospital medical officers, they treat about 6000 inpatients and 9000 emergency department patients annually. Another 14 specialists visit the town regularly to treat patients at the hospital.

The components of the risk management program (Box 1) evolved over time. In 1989, the hospital medical staff chose four doctors to be medical reviewers and to form a surveillance committee. This committee was expanded to include a nurse in 1995 and a clinical risk manager in 1997.

Detection of adverse events

Inpatient medical record review: The medical records of all patients admitted to the hospital between July 1991 and September 1999 were reviewed shortly after discharge, using a process described previously.³ Briefly, each medical record was screened by medical records staff using eight general patient outcome criteria (Box 2). Records with at least one of these criteria were sent to one of the four nominated doctors to determine if an adverse event was present. This was defined as "an untoward patient event which, under optimal conditions, is not a consequence of the patient's disease or treatment".¹ The reviewer independently completed an adverse event analysis form for discussion at bimonthly meetings of the surveillance committee. Recommendations for action relating to patient care were made by the committee and forwarded to medical and nursing staff groups in the hospital. When a clinical risk manager was appointed, the adverse event analysis forms were first forwarded to the manager to determine if immediate action was required.

Emergency department medical record review: The medical records of all patients who attended the emergency department between October 1997 and September 1999 were reviewed. An administrative database of all inpatient admissions and emergency department attendances was screened for five general patient outcome criteria (Box 2), using software designed for the study. Attendances that screened positive were reviewed by the hospital's clinical risk manager and, if an adverse event was detected, by the director of medical services. If an event was confirmed, it was further analysed and recommendations to prevent its recurrence were made to relevant hospital staff, using the same committee review process as used for inpatient records.

Clinical incident reporting: A clinical incident reporting system was developed in 1997 by the Australian Patient Safety Foundation, an independent organisation in Adelaide that promotes patient safety. Hospital staff in Horsham were educated about the system and encouraged to report clinical incidents and "near-misses". A clinical incident was defined as "any event that has caused harm, or has the potential to harm, a patient, visitor or staff member, or any event which involves malfunction, damage or loss of equipment or property, and any event which might lead to a complaint".⁵

Incident reporting forms developed by the Foundation were placed in all departments. These forms comprise two parts: the first provides details of actual or potential clinical incidents, while the second allows the incident to be reported anonymously to a national database. Staff members reporting incidents could identify themselves or remain anonymous. Completed forms for incidents reported between October 1997 and September 1999 were sent to the clinical risk manager for local analysis and were also reported to the national database.

General practitioner reporting: As adverse events related to inpatient care may occur or be recognised after patient discharge from hospital, an adverse event reporting form was included in the inpatient summary routinely sent to each patient's general practitioner (GP). GPs were asked to attach the form to the patient's medical record for a month and to complete and return the form if they detected an adverse event.

External sources: Some adverse events occur rarely in individual hospitals. Details about serious but infrequent adverse events at other hospitals were obtained from coronial and consultative committee reports, insurers, medical indemnity organisations, medical and nursing journals and the media. If the surveillance committee thought the event could occur locally, action was taken to reduce the risk.

Patient satisfaction: Patient perspective on adverse events was sought through patient satisfaction surveys, focus groups and patient complaints. Satisfaction surveys were posted to every 10th patient who attended the emergency department or was admitted to the hospital.

Event analysis and action

When an adverse event was detected, its likelihood and consequences were estimated in accordance with the Australia/ New Zealand Risk Management Standard⁶ (Box 3). Events were ranked according to their risk severity (risk severity = consequence score x likelihood score) (Box 4). Events with high risk severity were given priority for analysis, and action was taken to reduce the risk, as described previously⁷ (Box 5). For adverse events with low risk priority, the surveillance committee decided whether to take action or to accept the risk and continue monitoring for that event.

All data from the inpatient adverse event analysis forms were entered into a database program developed from the Clipper database compiler software package.⁸ Data from emergency department and GP reports were entered into access databases,⁹ and clinical incidents were entered into the Australian Patient Safety Foundation's database.⁴

Statistical analysis

The

² test was used for categorical comparisons of data. A P value < 0.05 was considered to indicate statistical significance; all tests were two-tailed. Statistical analyses were performed using the statistical package GraphPad Instat.¹⁰ Confidence intervals were calculated using standard methods.¹¹

Results

Inpatient medical record review

A total of 49 834 inpatients were discharged from the hospital between July 1991 and September 1999. The medical records of 4199 (8.43%) screened positive for one or more of eight general patient outcomes, and 386 (0.77%) contained an adverse event. These events were analysed and action was taken to reduce the probability of recurrence. The annual rate of adverse events decreased between the first and eighth years of the study from 1.35% of all patients discharged (69 events) to 0.74% (49 events) (

² = 31.31; df = 7; P < 0.001). This trend was linear (

² =11.52; df = 1; P < 0.001) (Box 6). The absolute risk reduction was 0.61% (95% CI, 0.23%-0.99%), and the relative risk reduction was 44.9% (95% CI, 16.9%-72.9%).

Emergency department medical record review

A total of 20 050 patients attended the emergency department between October 1997 and September 1999. The medical records of 544 screened positive for one or more of five general patient outcomes (2.71% of all patient attendances), and 250 (1.24%) contained an adverse event. Action was taken to reduce the probability of recurrence. The quarterly rate of adverse events decreased between the first and eighth quarters of monitoring from 3.26% of all attendances (84 events) to 0.48% (12 events) (

² = 120.43; df = 7; P < 0.001). The trend was linear (

² = 87.64; df = 1; P < 0.001) (Box 6). The absolute risk reduction was 2.78% (95% CI, 2.04%-3.52%), and the relative risk reduction was 85.3% (95% CI, 62.7%-100%).

Clinical incident reporting

Between October 1997 and September 1999, hospital staff completed 621 clinical incident forms, and 66 adverse events were found. The most common reported incidents were patient falls (280 incidents, 45% of all reported incidents) and medication errors (93; 15%). In response to the number of falls, a falls risk assessment tool was developed. Each patient over 65 years of age underwent a falls risk assessment on admission to hospital. Subsequently, the number of patient falls resulting in fractures while in hospital decreased.

The 66 events detected by clinical incident reporting made up 16.3% of the total of 405 adverse events detected between October 1997 and September 1999 by clinical incident reporting and medical record review; 250 events (61.7%) were detected by emergency department medical record review and 89 (22.0%) by inpatient medical record review. Four adverse events were detected by more than one method; these were associated with failure of equipment in the operating room and an inpatient fall. For analysis, these four events were allocated to the first method by which they were detected.

General practitioner reporting

Between January and September 1999, 21 reports were made by general practitioners (0.25% of patients discharged). An adverse event was identified in 16 (76% of all reports). Events included discharge medication errors, postoperative wound infections and other surgical complications. Three patients required readmission.

External sources

Information about 12 adverse events at other hospitals in Victoria was obtained from the media and coronial reports. After analysis, preventive administrative and clinical changes were implemented in Horsham. These included the introduction of an organisation-wide policy on equipment service contracts, installation of thermostatic mixing valves in the hot water system to reduce burns, and development of an intercostal catheter insertion policy.

Patient satisfaction

Of the 69 formal complaints received by the hospital between October 1997 and September 1999, 11 (16%) related to clinical care. On review, four were associated with an adverse event. Complaints were analysed in the same way as adverse events from other sources.

Discussion

Our study has demonstrated that adverse events in hospitals can be detected using medical record review, clinical incident reporting and other methods. The rate of adverse events can then be reduced using a systems approach to event analysis, followed by appropriate action and continued monitoring for adverse events to evaluate the effectiveness of the action.

As we found that few individual events were identified by more than one detection method, the use of multiple detection methods increased the total number of events identified. Scoring the risk associated with each event, using its likelihood and consequences, allowed events found by different methods to be ranked and prioritised for action to reduce risk. This allowed available resources to be directed to events with the greatest patient risk.

To our knowledge, this is the first comparative study over time of a clinical risk management program that used diverse methods to detect adverse events and reduce their rate. Other studies have used a single detection method to measure adverse event rate at one point in time. For example, inpatient medical record review was used in the multihospital Harvard Medical Practice Study¹ and the Quality in Australian Health Care Study.² These studies detected adverse event rates of 3.7%¹ and 16.6%² of hospital admissions, respectively. Both used 18 screening criteria, including some that required clinical judgement, and neither measured the rate of adverse events over time after intervention. Our study found a lower rate of inpatient adverse events, but used only eight screening criteria, none requiring clinical judgement. We are not aware of any comparative studies that measured the adverse event rate in hospital emergency departments.

The rate of adverse events found using clinical incident reporting depends on the rate of reporting. For example, an increase in reports of medication errors may reflect an increase in errors, the rate of reporting, or both. Reporting rates vary greatly between hospitals, making meaningful comparisons difficult.

In a study in the United States comparing adverse events reported by resident medical staff and those found by medical record review, 30.8% of adverse events were found by both methods.¹² This is a much greater proportion than the 0.9% of events found by both methods in our study. However, in our study, most events were reported by nursing staff, with medical staff reporting few events.

Although we used multiple methods to detect adverse events, we did not find all adverse events that occurred in the hospital. Although we could have found more by using more screening criteria, our use of five to eight screening criteria for medical record review, rather than the 18 used in some other studies,^1,2 meant that this would have required more resources. The study did not include a control hospital where adverse events were detected but no analysis was performed nor action taken. Also, adverse event rates were not adjusted for patient severity. Therefore, other factors may explain the reduction in adverse events.

In our study, resource limitations meant that medical records identified by screening were reviewed by a single reviewer. In the Harvard Medical Practice Study and the Quality in Australian Health Care Study, each medical record identified by screening was reviewed by two reviewers and, if they disagreed, by a third. The resources for such intensive review of records are unlikely to be available in most hospitals.

The strengths of our study included its eight-year duration and prospective nature. By keeping the method constant (eg, number and types of screening criteria and three of the four reviewers), the rate of adverse events could be meaningfully compared over a long period, and the effects of actions assessed.

Further research is required to improve methods of detecting adverse events. A greater proportion of events might be detected with more effective and efficient screening criteria. A potentially effective criterion would be a code for external cause of injury¹³ to be assigned by medical records staff when coding diagnoses. Another potential screening device is clinical pathways that can detect adverse events by analysing deviations from the pathways. The development of electronic records may also assist in detecting adverse events. In addition, national databases of adverse events reported as clinical incidents or from coronial inquests would provide further information to reduce risk.

Actions that are effective in changing clinical behaviour have been discussed in detail previously.⁷ Although some effective strategies are available, changing health delivery systems and clinical behaviour is frequently complex and difficult, and many strategies in use are not effective.¹⁴ More research is required to develop additional effective strategies.

The components of the risk management program used in our study could be applied in hospitals of varying sizes. Hospitals or individual departments can decide which detection methods are appropriate for their services and available resources. For example, medical record screening does not need to use all outcome criteria, and further program components can be added over time.

Finally, we believe that the risk management program has allowed our patients to receive better care with fewer adverse events and has been an effective use of resources.

Acknowledgements

We wish to thank the staff of the Wimmera Health Care Group for their enthusiastic participation in the program, especially Mrs Cathy Dooling, Manager Health Information Services, and staff, and previous members of the surveillance committee. This program was partly funded by a grant from the Victorian Department of Human Services.

References

Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalised patients: results of the Harvard Medical Practice Study 1. N Engl J Med 1991; 324: 370-376.
Wilson RM, Runciman WB, Gibberd RW, et al. The Quality in Australian Health Care Study. Med J Aust 1995; 163: 458-471.
Wolff AM. Limited adverse event screening: using record review to reduce hospital adverse patient events. Med J Aust 1996; 164: 458-461. <eMJA full text>
Australian Patient Safety Foundation. What is incident monitoring? The Australian Incident Monitoring Study. Adelaide: The Foundation, 1997.
Leape LL. Error in medicine. JAMA 1994; 272: 1851-1857.
Standards Australia. Australian/New Zealand Standard 43:60. Sydney: Standards Association of Australia, 1999.
Wolff AM, Bourke J. Reducing medical errors: a practical guide. Med J Aust 2000; 173: 247-251.
Clipper. Version 5.0. Los Angeles, Calif: Nantucket Corporation, 1990.
Microsoft access. Relational database management system for windows. Version 7.0. Seattle, Wash: Microsoft Corporation, 1999.
GraphPad Instat. Version 2.0. San Diego, Calif: GraphPad Software, 1992.
Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology: a basic science for clinical medicine. 2nd ed. Boston: Little Brown and Co, 1991: 218.
O'Neil AC, Peterson LA, Cook EF, et al. Physician reporting compared with medical-record review to identify adverse medical events. Ann Intern Med 1993; 119: 370-376.
O'Hara DA, Carson NJ. Reporting of adverse events in hospitals in Victoria, 1994-1995. Med J Aust 1997; 166: 460-463.
Oxman AD, Thomson MA, Davis DA, Hayes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ 1995; 153: 1423-1431.

(Received 23 Oct 2000, accepted 6 Feb 2001)

Authors' details

Clinical Risk Management Unit, Wimmera Health Care Group, Horsham, VIC.
Alan M Wolff, FRACGP, MBA, Director of Medical Services, and Director of Accident and Emergency Department;
Jo Bourke, RN, GradDipCM, Clinical Risk Manager;
Ian A Campbell, FRACS, Visiting General Surgeon;
David W Leembruggen, FRACGP, Visiting General Practitioner, and Director of Postgraduate Education.

Reprints: Dr A M Wolff, Medical Administration, Wimmera Health Care Group, Baillie Street, Horsham, VIC 3400.
whcgmedATnetconnect.com.au