Limited adverse occurrence screening: using medical record review to reduce hospital adverse patient events
Alan M Wolff
Readers may print a single copy for personal use. No further reproduction or distribution of the articles should proceed without the permission of the publisher. For permission, contact the Australasian Medical Publishing Company
Journalists are welcome to write news stories based on what they read here, but should acknowledge their source as "an article published on the Internet by The Medical Journal of Australia <http://www.mja.com.au/>".
- - ©MJA1997
Objectives: To determine whether continuous
detection of adverse patient occurrences followed by analysis and
medical intervention can alter the rate of adverse occurrences.
Design and participants: 15 912 patients discharged from one hospital were reviewed in two stages. Medical records were screened retrospectively by medical records staff for one or more of eight general patient outcome criteria. Those that screened positive for the criteria were reviewed by one of four doctors. If an adverse occurrence was confirmed, further analysis and recommendations for action to prevent its recurrence were made at meetings of the four doctors, and forwarded to a committee of visiting medical officers who decided on the appropriate course of action.
Setting: A rural base hospital in Horsham, Victoria, between July 1991 and June 1994.
Main outcome measures: The rate and severity of adverse patient occurrences in each year.
Results: 1465 records were screened positive for one or more criteria, and an adverse patient occurrence was confirmed in 155. 88 cases were determined to be minor or not preventable and further action (mostly by changes to hospital policies) was recommended for the remaining 67. Over the three years, the number of adverse occurrences fell from 69 (1.35% of all patient discharges in the first year) to 33 (0.58% of all patient discharges in the third year) ( P < 0.0001) and there was no significant change in severity.
Conclusions: The rate of adverse patient occurrences can be significantly reduced by their continuous detection using retrospective screening in conjunction with review, analysis and action to prevent recurrences.
The objective of medical quality assurance programs is to improve the
quality of care provided to patients. However, "quality medical
care" has proved difficult to define, leading some programs to
attempt to detect "disquality", or events that should not happen
under optimal conditions.1
Quality, then, may be defined as the absence or lowering of the rate of
adverse patient events or occurrences.
Many preventable adverse patient events occur in hospitals. The Quality in Australian Health Care Study, which reviewed over 14 000 patient admissions in 28 hospitals in New South Wales and South Australia, found 16.6% involved an adverse event; half of these were assessed as highly preventable.2 The Harvard Medical Practice Study, a review of over 30 000 inpatient medical records from 51 acute-care hospitals in the United States, showed that adverse events occurred in 3.7% of hospitalisations and that many of these were the result of substandard care.3 Both these studies used methods that required substantial resources in time and money to detect adverse patient events, and neither looked at the effect of intervention in preventing recurrence of an adverse event.
Limited adverse occurrence screening is a continuous process of retrospective screening and review of inpatient medical records to detect adverse patient occurrences.4 It is referred to as "limited" because only eight general patient outcome criteria are used. When an adverse occurrence is found, appropriate action is taken to prevent its recurrence and the success of the action is determined by ongoing monitoring.4 Thus, the quality assurance feedback loop is securely closed. This study was designed to determine whether limited adverse occurrence screening could reduce the number and rate of adverse occurrences and therefore improve the quality of patient care.
The study was undertaken at Wimmera Base Hospital in Horsham, 300 km
north-west of Melbourne. There are 11 specialists and 14 general
practitioners on the medical staff who treat between 5000 and 6000
inpatients per year.
Four doctors were chosen by the Hospital Medical Staff Group to be medical reviewers and to form a Patient Care Committee. All doctors held dual positions in the hospital and had postgraduate clinical qualifications. They were a physician (also Director of Intensive Care), a surgeon (also Chairman of the Medical Staff Group), a general practitioner (also Director of Postgraduate Education) and the Medical Director (also Director of the Accident and Emergency Department).
The medical records for all inpatients discharged between 1 July 1991 and 30 June 1994 were screened by medical records staff using eight general patient outcome criteria (Box 1, below). An adverse patient occurrence analysis form was attached to any medical record that met one or more of the criteria. The record was reviewed by the doctor allocated to that criterion, who evaluated the care given to determine if an adverse patient occurrence had arisen before or during admission, and then completed the form. Adverse occurrences for which patients had had previous admissions were excluded.
1: General patient outcome criteria used for screening of medical records
An adverse patient occurrence was defined as "an untoward patient event which, under optimal conditions, is not a natural consequence of the patient's disease or treatment".1 I have described the limited adverse occurrence screening method in detail in an earlier paper.4
Medical care was evaluated using a six-point scale,5 to help the reviewer decide whether an adverse patient occurrence was caused by medical management: 1 = little or no evidence, 2 = slight evidence, 3 = not likely (less than 50:50 odds but a close call), 4 = more likely than not (greater than 50:50 odds but a close call), 5 = strong evidence, 6 = virtually certain. A score of four or more on this scale was regarded as an adverse patient occurrence. Severity of adverse occurrences was graded on a seven-point scale:1 0 = minor severity, 1 = minor temporary, 2 = minor permanent, 3 = major temporary, 4 = major permanent, 5 = potential major or major continuing, 6 = death.
The medical reviewer wrote brief notes about the case for the Patient Care Committee. Adverse occurrences were discussed at the Committee's bimonthly meetings and recommendations for further action relating to patient care were made and forwarded to the Medical Staff Group, which consisted of all visiting medical officers in the hospital and the Director of Medical Services.
All data from the adverse patient occurrence analysis forms were entered into a database program developed from the Clipper Dbase Compiler software package.6 A chi-squared test was used for statistical analysis, and confidence intervals were calculated for odds ratios using standard methods.7
A total of 15 912 inpatients were discharged between 1 July 1991 and 30
June 1994. Using the eight general patient outcome criteria, 1465
records (9.21%) were screened positive for one or more criteria by
medical records staff. The commonest criteria among these
were unplanned readmission of a patient within 28 days of discharge
(3.4% of all records screened) and transfer of a patient to another
acute-care facility (2.8% of all records screened) (Box 2). A medical
record was most likely to be subsequently confirmed as containing an
adverse patient occurrence when it was selected by the criterion
"return to operating theatre within seven days" (odds ratio =
3.98; 95% confidence interval, 2.15-7.37) (Box 2).
After medical review, 155 records (0.97% of all records screened) were found to contain an adverse patient occurrence using the six-point scale (Box 3). Of these adverse occurrences, 45 were minor and 110 were major (Box 4).
Of the 155 adverse patient occurrences confirmed on medical review, no further action was recommended by the Patient Care Committee in 88 cases (56.8%) (i.e., the adverse occurrence was considered to be not preventable or of minor significance). Recommended action for the other cases included changing the relevant hospital policy; presenting the case at a postgraduate meeting; undertaking a quality assurance program to investigate the adverse occurrence in detail; discussion with, or counselling of, the doctor involved; and, rarely, review of the doctor's clinical privileges or reporting the case to the hospital's insurers.
Over the three-year period of the study, 66 recommendations were made by the Patient Care Committee and almost all were accepted by the Medical Staff Group and became hospital policy. Changes in hospital policies were both clinical and administrative. Clinical policy changes included restricting some drug prescribing; revised protocols for reporting vital signs; eliminating use of multidose drug vials; formulating guidelines regarding fitness for general anaesthesia; and developing protocols for managing patients with alcohol withdrawal, haematemesis and melaena, and for patients who have had a cerebrovascular accident or who require analgesia.
Administrative policy changes included sending copies of referral letters when patients were transferred to other hospitals; sending private antenatal notes to the obstetric ward before the patient's confinement; redesigning the preadmission form to allow consent for procedures to be obtained in the consultant's room; and routinely including a copy of the death certificate in the medical record (if the patient had died in hospital). (Examples of the types of adverse occurrences, more detailed recommendations for further action and their subsequent reduction in incidence are given in an earlier paper.4)
The adverse patient occurrences were grouped by year to determine whether the actions instigated had any effect on the subsequent rate of adverse occurrences. The rate of adverse patient occurrences fell from 69 in the first year (1.35% of all patient discharges in that year) to 33 in the third year (0.58% of all patient discharges in that year) (Box 4). The proportion of adverse occurrences fell significantly with time (chi-squared = 17.11; df = 2; P = 0.0002) and this trend was linear (chi-squared = 16.87; df = 1; P< 0.0001).
When the severity rating of adverse events was divided into minor (severity score 0-2) and major (severity score 3-6), there was no significant change in the severity of adverse patient occurrences between the first, second and third years of the study (Box 4). Also, the proportion of adverse patient occurrences detected in each year by screening did not alter significantly over time for patient age, sex or insurance status (Box 5). The commonest major diagnostic categories, using the ICD-9 classification,8 in which patients had adverse occurrences were injury and poisoning, followed by digestive and circulatory disorders (Box 6, below).
Patient characteristics such as sex and age did not change significantly during the three years of the study, but the proportion of inpatients privately insured fell from 26.6% to 24.3% (chi-squared = 10.24; df = 2; P< 0.01).
Limited adverse occurrence screening has been shown to be effective
in detecting approximately 50% of adverse patient occurrences,9 a much higher
proportion than found by traditional medical quality assurance
programs. This study has shown that it is also possible to
subsequently reduce the number of adverse occurrences by more than
50% over a three-year period. These results indicate that use of a more
complex and expensive screening system is not required to achieve a
significant reduction in adverse occurrences.
Adverse occurrence screening has been shown to be valid and reliable, with a high level of agreement between reviewers as to whether a patient has had an adverse occurrence.10 Although in this study each medical record was reviewed by only one doctor, other studies using a similar methodology and rating scale have shown high agreement.5 Limited adverse occurrence screening is efficient as it requires the review of the medical records of slightly less than 10% of all patients discharged, costs only 0.1% of an acute-care hospital's total budget, and is fast and accurate (false positive rate, 2.0%; false negative rate, 0.4%).9
In this study, 9.21% of medical records were screened positive for one or more of the eight general patient outcome criteria and an adverse occurrence was confirmed in 0.97%. Little or no clinical judgement was required in the screening process. This compares with a rate of 8.4% (confirmed in 0.73%) in a study using similar screening criteria in a medium-sized New South Wales hospital.11 In the Quality in Australian Health Care Study, potential adverse events were detected in 43.7% of the medical records on initial screening (a much higher rate than in other studies) using 18 criteria, some of which required clinical judgement.2 Adverse events were confirmed in 16.6%, although, unlike in this study, 49% of these occurred before the sample admission. An average rate of potential adverse events of 13.2% was found in a study of 146 Veterans Affairs Medical Centers in the United States that used nine screening criteria.12 There was considerable variation between centres, with a 25th percentile of 8.6% (confirmed in 1.7% of records) and a 75th percentile of 27.2% (confirmed in 7.9% of records). In the Harvard Medical Practice Study, potential adverse events were detected on initial screening in 25.9% of medical records using 18 screening criteria.13 Adverse events were confirmed in 0.2% to 7.9%, depending on hospital complexity and location, with the rate in rural hospitals being 1%.13
Limited adverse occurrence screening can be adapted to all sizes of hospital. It has recently been introduced into 10 small hospitals in central western Victoria by the Division of General Practice in that area working with the Monash University Centre for Rural Health in Moe, Victoria. In larger hospitals, limited adverse occurrence screening could be introduced on a departmental basis.
The program could be enhanced by combining its retrospective medical record review with prospective critical incident reporting. In the United States, adverse occurrence screening and incident reporting both detected a substantial number of adverse events; however, there was less than a 50% overlap in the adverse events detected by each method when used concurrently.14 Thus, a program that simultaneously uses both methods may detect the largest pool of adverse occurrences for analysis and development of preventive strategies.
|I wish to thank Mr Ian Campbell, Dr David Leembruggen and Dr Grant Phelps for their enthusiastic participation in the Patient Care Committee; Mrs Cathy Dooling, Chief Medical Records Administrator, and her staff, for their support with the screening program; Mr Kieran Loughran, Computer Systems Officer, for assistance with data collection; and Mrs Naomi Uytdehaag for preparing the manuscript.
(Received 2 Sep 1995, accepted 2 Feb 1996)
Authors' detailsWimmera Base Hospital, Horsham, VIC.
Alan M Wolff, FRACGP, MBA, Director of Medical Services, and Director of the Accident and Emergency Department.
Reprints: Dr A M Wolff, Medical Administration, Wimmera Base Hospital, Baillie Street, Horsham, VIC 3400.
© 1997 Medical Journal of Australia.
Received 2 March 2024, accepted 2 March 2024