Connect
MJA
MJA

The Threats to Australian Patient Safety (TAPS) study: incidence of reported errors in general practice

Meredith A B Makeham, Michael R Kidd, Deborah C Saltman, Michael Mira, Charles Bridges-Webb, Chris Cooper and Simone Stromer
Med J Aust 2006; 185 (2): 95-98. || doi: 10.5694/j.1326-5377.2006.tb00482.x
Published online: 17 July 2006

Abstract

Objective: To determine the incidence of errors anonymously reported by general practitioners in NSW.

Design: The Threats to Australian Patient Safety (TAPS) study used anonymous reporting of errors by GPs via a secure web-based questionnaire for 12 months from October 2003.

Setting: General practices in NSW from three groupings: major urban centres (RRMA 1), large regional areas (RRMA 2–3), and rural and remote areas (RRMA 4–7).

Participants: 84 GPs from a stratified random sample of the population of 4666 NSW GPs — 41 (49%) from RRMA 1, 22 (26%) from RRMA 2–3, and 21 (25%) from RRMA 4–7. Participants were representative of the GP source population of 4666 doctors in NSW (Medicare items billed, participant age and sex).

Main outcome measures: Total number of error reports and incidence of reported errors per Medicare patient encounter item and per patient seen per year.

Results: 84 GPs submitted 418 error reports, claimed 490 864 Medicare patient encounter items, and saw 166 569 individual patients over 12 months. The incidence of reported error per Medicare patient encounter item per year was 0.078% (95% CI, 0.076%–0.080%). The incidence of reported errors per patient seen per year was 0.240% (95% CI, 0.235%–0.245%). No significant difference was seen in error reporting frequency between RRMA groupings.

Conclusions: This is the first study describing the incidence of GP-reported errors in a representative sample. When an anonymous reporting system is provided, about one error is reported for every 1000 Medicare items related to patient encounters billed, and about two errors are reported for every 1000 individual patients seen by a GP.

The importance of better understanding error and safety in primary care is widely accepted, with recent calls to promote efforts and improve resources in this area of research.1 A few studies have described errors in general practice using the incident monitoring technique,2-10 but none have quantified the problem.

A study of family medicine physicians in Cincinatti, USA, showed errors were identified in almost a quarter of clinical encounters.2 An error rate of 75.6 per 1000 general practice appointments (7.6%) was reported in one UK study.3 Reviewing incident reports from an outpatient primary care setting, a US study reported a prevalence of adverse events of 3.7 per 100 000 clinic visits.11

The Threats to Australian Patient Safety (TAPS) study is the first to estimate the incidence of errors reported by general practitioners in the community from a representative sample, and describes a new method for reporting errors using a secure web-based tool.

Methods
Sampling method

The General Practice Programs Branch of the Commonwealth Department of Health and Ageing (GP Branch) provided a stratified random sample of 320 names from the population of 4666 vocationally registered GPs in NSW whose Medicare billings in the previous quarter indicated that they were engaged in full-time work.14

A sample size of 80 was calculated from pilot study results of 134 reports made from up to 23 GPs over 4 months, which had a 25% participation rate.5 Allowing 1 week of absence each and estimating 132 consultations per week,15 we hypothesised a reported error rate of 0.27% per consultation. To estimate this with a 95% confidence interval of 0.25%–0.29%, we required 205 465 patient encounters. We anticipated that 80 GP participants would have about 480 000 patient encounters, with the excess allowing for any effect of clustered samples.

The sample was drawn from Rural, Remote and Metropolitan Area (RRMA) groupings of RRMA 1, RRMA 2–3, and RRMA 4–7.14 The GP Branch stratified the sample by gender and age, dichotomised around 45 years. One hundred and sixty (50%) of the sample were drawn from RRMA 1, 80 (25%) from RRMA 2–3 and 80 (25%) from RRMA 4–7 regions. The proportions of the source population in these regional groupings were 68%, 16% and 16%, respectively. The smaller groups were over-sampled to ensure adequate counts of Medicare items to allow future subgroup analysis.

A research assistant made initial contact with the GPs by telephone. If interest was expressed, an information pack was sent, followed by further telephone contact. Reasons for non-participation were recorded. All GPs who commenced the study were visited by a research assistant. Informed consent was obtained, the definition of “error” was explained, and a demonstration of the website to be used for reporting was given.

Anonymous online error reporting

A secure website and reporting process were developed to house the error questionnaire, shown in Box 1, which was piloted in 2001.5 Data transmission was protected by 128-bit encryption and the website was hosted by a secure server behind a firewall. The website allowed participants to contact investigators and view linked medical resources. Participants were asked about any website problems in monthly contact via email, telephone or facsimile.

A self-chosen Personal Identification Number (PIN) and the RRMA grouping were required to submit a report. Participants submitted a blank report before the start of the study for website testing. The participant’s PINs were unknown to investigators, which protected reporter anonymity.

Results
Number of error reports submitted

There were 418 TAPS reports from 85 PINs describing an error during the study. The mean number of error reports per month was 34.8 (range, 15–60). At feedback interview, 79 of the 82 completing participants had submitted a report.

Box 2 shows RRMA grouping comparisons of the number of reports submitted, PINs submitted, GPs who reported at interview sending a report, average number of reports per GP, and the median number of reports per PIN over the study duration.

Frequency of error reporting

The distribution of reporting frequencies among RRMA groupings was assessed by looking at the number of reports per PIN, shown in Box 3. This varied from one to 25, with a median of three for all participants combined. There was no significant difference between RRMA groupings in the median number of reports submitted per PIN (P = 0.40) (for this analysis, values that were equal to the median were evenly split between the above and below groups).

Discussion

Original research on errors and adverse events in primary care consists of relatively limited numbers of studies, focusing on qualitative descriptions.16 The TAPS study provides the first calculated incidence of reported error from a representative random sample of GPs. Despite the small sample size and participation rate of 26%, these findings are likely to be generalisable to GPs in NSW, as we detected no statistically significant differences between the age, sex, and Medicare billings when comparing participants to the source population of 4666 GPs. Comparative data were not available from the GP Branch for the non-participants due to consent issues.

We conclude from our incidence findings that, when an anonymous and simple reporting system is provided, about one error is reported for every 1000 Medicare items related to patient encounters billed per year, and about two errors are reported for every 1000 individual patients seen by a GP per year.

It has been suggested previously that an error database in general practice should be established.17,18 Our results indicate that using a secure website with an anonymous reporting method is a practical way of collecting error information from GPs, as 94% of doctors who enrolled in the study reported at interview that they sent an error report.

Our findings provide an estimation of the incidence of errors in general practice that would be reported if an appropriate reporting system were in place, but this should not be interpreted as an estimate of the total number of errors that participants noticed, or of the underlying number of errors occurring in the GP community. It is very difficult to assess the proportion of errors that would go unreported even when a reporting system such as TAPS is available. A GP may not be aware that an error has occurred. GPs have been found to under-report adverse drug events,19 and so other patient safety threats may also be under-reported.

Anonymity was of major importance in our methodological design, to encourage reporting. There has previously been a reluctance of health professionals to address the problem of errors due to feelings of guilt and a desire to avoid disapproval from colleagues.20 However, using a PIN that was unknown to investigators led to some difficulties in interpreting reports per GP. Researchers were unable to assist if a PIN was forgotten, and there was no way of knowing how many GPs changed PINs during the study. Seventy-nine GPs gave feedback that they submitted an error report, but 85 PINs were used for error reports, which suggests that on at least six occasions GPs changed their PINs. Our estimate of report numbers per participant, which is based on PINs, could therefore be an underestimate.

The number of individual Medicare items billed relating to a patient encounter is the best approximation that we could make of separate consultations, but should not be interpreted directly as such. A GP may claim more than one Medicare item from a single encounter, which is not revealed in our data. Consultations may also have occurred which were not billed, charged to the patient privately, or charged to the Department of Veterans’ Affairs, which keeps separate data from the GP Branch.

To our knowledge, there are no studies directly comparable with our incidence results, which are lower than hypothesised. The authors of a US study that found an error rate close to 25% state that their results were not generalisable due to limitations of the study.2 Only a small group of 15 doctors in seven practices made reports, completing a review form after every consultation during three half-day sessions. The UK finding of a 7.6% error rate was based on a 2-week collection of errors from 10 practices in a single city.3 However, a major difference is that this result was based upon a collection of reports from a large cross-section of staff working in general practice, with 163 people able to submit reports, and a high proportion of participants who had administrative rather than clinical roles.

Our method of prospective self-reporting appears to have returned a higher rate of error detection than that in a US study using retrospective record review in eight primary health care clinics.11 Again, direct comparison is difficult, as medical adverse events were counted in this work, rather than errors in the sense that we defined.

In considering the adoption of a method of error reporting for primary care, retrospective record review lacks potential for use as an active learning tool. A prospective electronic reporting tool could be enhanced with the addition of feedback to the user and links to educational activities, plus have the scope to allow many other stakeholders in primary care settings to contribute error information.

The TAPS study improves our current understanding of the incidence of reporting threats to patient safety in general practice, and its method provides a way for GPs to discuss errors in a non-threatening environment.

1 Error questionnaire used in the Threats to Australian Patient Safety (TAPS) study

Question

Answer format


a) What is the RRMA area of your practice?

RRMA 1, 2–3 or 4–7

b) Is the problem related to a specific patient?

Yes/No

c) How well do you know the patient?

5-point Likert scale

d) What is the patient’s age?

Free text

e) What is the patient’s sex?

Male/Female

f) Is the patient from a non-English speaking background?

Yes/No

g) Is the patient of Aboriginal or Torres Strait Islander descent?

Yes/No

h) What happened? Please consider what, where and who was involved.

Free text

i) What was the result? Please think about actual and potential consequences.

Free text

j) What may have contributed to this error? Please consider any special circumstances.

Free text

k) Where did the error happen? Choose all that apply from Office or surgery, Nursing home, Hospital, Patient’s home, Telephone contact, Emergency Room, Laboratory, Pharmacy, Radiology

Check a box or boxes: nine choices

l) To your knowledge, was any patient harmed by this error?

Yes/No

m) If yes, how would you rate the seriousness of this harm?

5-point Likert scale

n) How often does this error occur in your practice? First time, Seldom 1–2 per year, Sometimes 3–11 per year, Frequently >1/month

Check a box: four choices

o) What could have prevented this error? Please consider what could be done to prevent similar episodes from happening in the future.

Free text

p) Other comments?

Free text


RRMA = Rural, Remote and Metropolitan Area.

  • Meredith A B Makeham1
  • Michael R Kidd1
  • Deborah C Saltman1
  • Michael Mira1
  • Charles Bridges-Webb2
  • Chris Cooper1
  • Simone Stromer1

  • 1 Discipline of General Practice, The University of Sydney, Sydney, NSW.
  • 2 RACGP NSW Projects, Research and Evaluation Unit, The Royal Australian College of General Practitioners, Sydney, NSW.


Correspondence: makeham@ozemail.com.au

Acknowledgements: 

Meredith Makeham is a National Health and Medical Research Council scholarship holder. We gratefully acknowledge the contribution of the 84 NSW GPs who provided the data, and the funding from the National Health and Medical Research Council and the Primary Health Care Research, Evaluation and Development program. We also thank Mr Tony Lawrence and Ms Angela Mikalauskas of the General Practice Branch, Commonwealth Department of Health and Ageing, for assistance with sampling and Medicare data provision, and Ms Geraldine Card for her valuable contribution in managing the study and collating results.

Competing interests:

None declared.

  • 1. Jacobson L, Elwyn G, Robling M, Jones RT. Error and safety in primary care: no clear boundaries. Fam Pract 2003; 20: 237-241.
  • 2. Elder NC, Vonder Meulen M, Cassedy A. The identification of medical errors by family physicians during outpatient visits. Ann Fam Med 2004; 2: 125-129.
  • 3. Rubin G, George A, Chinn DJ, Richardson C. Errors in general practice: development of an error classification and pilot study of a method for detecting errors. Qual Saf Health Care 2003; 12: 443-447.
  • 4. Dovey SM, Phillips RL, Green LA, Fryer GE. Types of medical errors commonly reported by family physicians. Am Fam Physician 2003; 67: 697.
  • 5. Makeham MA, Dovey SM, County M, Kidd MR. An international taxonomy for errors in general practice: a pilot study. Med J Aust 2002; 177: 68-72.
  • 6. Dovey SM, Meyers DS, Phillips RL Jr, et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care 2002; 11: 233-238.
  • 7. Steven ID, Malpass A, Moller J, et al. Towards safer drug use in general practice. J Qual Clin Pract 1999; 19: 47-50.
  • 8. Bhasale A. The wrong diagnosis: identifying causes of potentially adverse events in general practice using incident monitoring. Fam Pract 1998; 15: 308-318.
  • 9. Bhasale AL, Miller GC, Reid SE, Britt HC. Analysing potential harm in Australian general practice: an incident-monitoring study. Med J Aust 1998; 169: 73-76.
  • 10. Britt H, Miller GC, Steven ID, et al. Collecting data on potentially harmful events: a method for monitoring incidents in general practice. Fam Pract 1997; 14: 101-106.
  • 11. Fischer G, Fetters MD, Munro AP, Goldman EB. Adverse events in primary care identified from a risk-management database. J Fam Pract 1997; 45: 40-46.
  • 12. Woolf SH, Kuzel AJ, Dovey SM, Phillips RL Jr. A string of mistakes: the importance of cascade analysis in describing, counting, and preventing medical errors. Ann Fam Med 2004; 2: 317-326.
  • 13. Tilyard M, Dovey S, Hall K. Avoiding and fixing medical errors in general practice: prevention strategies reported in the Linnaeus Collaboration’s Primary Care International Study of Medical Errors. N Z Med J 2005; 118: U1264.
  • 14. Budget and Performance Branch, Primary Care Division, Department of Health and Ageing. General Practice in Australia: 2004. 1st ed. Canberra: Commonwealth of Australia, 2005.
  • 15. General Practice Branch, Department of Health and Aged Care. General Practice in Australia: 2000. 1st ed. Canberra: Commonwealth of Australia, 2000.
  • 16. Elder NC, Dovey SM. Classification of medical errors and preventable adverse events in primary care: a synthesis of the literature. J Fam Pract 2002; 51: 927-932. [Erratum appears in J Fam Pract 2002; 51: 1079.]
  • 17. Sheikh A, Hurwitz B. Setting up a database of medical error in general practice: conceptual and methodological considerations. Br J Gen Pract 2001; 51: 57-60.
  • 18. Runciman WB. Lessons from the Australian Patient Safety Foundation: setting up a national patient safety surveillance system — is this the right model? Qual Saf Health Care 2002; 11: 246-251.
  • 19. Moride Y, Haramburu F, Requejo AA, Begaud B. Under-reporting of adverse drug reactions in general practice. Br J Clin Pharmacol 1997; 43: 177-181.
  • 20. Kidd MR, Veale B. How safe is Australian general practice and how can it be made safer [editorial]? Med J Aust 1998; 169: 67-68.

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Online responses are no longer available. Please refer to our instructions for authors page for more information.