Connect
MJA
MJA

Risks of complaints and adverse disciplinary findings against international medical graduates in Victoria and Western Australia

Katie Elkin, Matthew J Spittal and David M Studdert
Med J Aust 2012; 197 (8): 448-452. || doi: 10.5694/mja12.10632
Published online: 15 October 2012

Abstract

Objective: To determine whether international medical graduates (IMGs) have more complaints made against them to medical boards and experience more adverse disciplinary findings than Australian-trained doctors.

Design and setting: Data on all complaints made against doctors to medical boards in Victoria and Western Australia over 7.5 years and 5.25 years, respectively, were extracted and linked with information on all doctors registered in those states over the same time periods. The data pertained to complaints resolved before February 2010 in Western Australia and June 2010 in Victoria, the dates of the respective extractions. We tested for associations between IMG status and the incidence of complaints using multivariable logistic regression.

Main outcome measures: Incidences of complaints and adverse disciplinary findings.

Results: Among 39 155 doctors registered in Victoria and Western Australia in the study period, 5323 complaints were made against 3191 doctors. Thirty-seven per cent of registered doctors were IMGs. The odds of complaints were higher against IMGs than non-IMGs (odds ratio [OR], 1.24; 95% CI, 1.13–1.36; P < 0.001), as were the odds of adverse disciplinary findings (OR, 1.41; 95% CI, 1.07–1.85; P = 0.01). However, disaggregation of IMGs into their countries of qualification showed wide variation: doctors who qualified in Nigeria (OR, 4.02; 95% CI, 2.38–6.77), Egypt (OR, 2.32; 95% CI, 1.77–3.03), Poland (OR, 2.28; 95% CI, 1.43–3.61), Russia (OR, 2.21; 95% CI, 1.14–4.26), Pakistan (OR, 1.80; 95% CI, 1.09–2.98), the Philippines (OR, 1.80; 95% CI, 1.08–3.00) and India (OR, 1.61; 95% CI, 1.33–1.95) had higher odds of attracting complaints, but IMGs from the 13 other countries examined had odds that were not significantly different from Australian-trained doctors.

Conclusions: Overall, IMGs are more likely than Australian-trained doctors to attract complaints to medical boards and adverse disciplinary findings, but the risks differ markedly by country of training. Better understanding of such heterogeneity could inform a more evidence-based approach to registration and oversight rules.

Australia has a shortage of doctors, particularly in rural areas.1,2 Like many other developed countries with this problem, its policy response has involved heavy reliance on the immigration of doctors whose primary medical qualification is from another country.3-5 Numbers of international medical graduates (IMGs) have grown to nearly 25% of doctors in Australia.1 The United States, United Kingdom, Switzerland, New Zealand, Sweden, Canada and Ireland have similar or larger proportions of IMGs in their medical workforces, and the prevalence of IMGs in Organisation for Economic Cooperation and Development countries has grown rapidly over the past decade.3

The effects on quality of care are unclear. Although some high-profile cases featuring incompetent IMGs have ignited public concerns,6,7 the evidence base is relatively thin and somewhat confusing. Previous international studies have compared IMGs to their domestically trained counterparts in relation to patient outcomes of care8,9 and risks of complaints and disciplinary action.10-17 Results from these studies run the gamut: most have found no association,9-14 but IMG status has also been found to be associated with higher15 and lower16 complaint-related risks. No research of this kind has been conducted in Australia.

Complaints to medical regulatory authorities are a useful, albeit imperfect, marker of quality.18-20 They capture both objective and subjective dimensions of patients’ care experiences and, unlike most outcome-based quality measures, can generally be examined at a “whole-of-population” level. In the IMG context, complaints to medical boards have the added advantage of being connected to the agencies that regulate doctors, which means that any heightened risks identified have immediate policy relevance.

We analysed a large sample of complaints lodged with medical boards in Victoria and Western Australia. Collectively, these two states account for about 37% of the nation’s registered doctors and 37% of IMGs.1 Our twin goals were to determine whether IMGs were disproportionately likely to attract complaints, and to analyse the extent to which that risk varied across countries of qualification. In addition, recognising that a variety of factors, including cultural misunderstandings and racism, may drive groundless complaints, we followed the complaints through the disciplinary process to test whether rates of adverse findings against IMGs and Australian-trained doctors differed.

Methods
Setting

Data for this study came from the Medical Practitioners Board of Victoria and the Medical Board of Western Australia before the incorporation of these boards into the national practitioner regulatory framework in mid-2010. We selected these two states because of their relatively high numbers of IMGs, the similarity of their disciplinary processes, and their diverse geographies and degrees of urbanisation.

Data collection

The boards maintained separate datasets for complaints and registrations. From the complaints datasets, we extracted information on complaints resolved between 2001 and 2010. Specifically, the sampling criteria were based on lodgement date (all complaints lodged between 1 July 2001 and 31 December 2008 in Victoria, and between 1 October 2003 and 31 December 2008 in Western Australia). The dates of the extractions were February 2010 in Western Australia and June 2010 in Victoria. Extracting the data in 2010 and fixing the end of the lodgement period at the end of 2008 allowed at least a year for the latest complaints to reach resolution (nearly all complaints are resolved within 12–18 months). From the registration datasets, we extracted information on all doctors registered to practise in the corresponding time periods. The Human Research Ethics Committee at the University of Melbourne approved the study.

Variables

The complaints data included information on the complainant, complaint date, type of complaint and its path through the disciplinary process. The registration data included information on registrants’ sex, medical specialty, registration type, practice location, and place and year of primary medical qualification.

The variable “registration type” refers to the form of registration the doctor held. We coded this variable into four categories: general, specialist (registration to work for a limited time in a specified specialty), area of need (registration to work in specified geographical areas with acute workforce shortages), and other (including provisional registration and registration granted for specific purposes, such as training and supervision or research).

Two variables had to be excluded from the analyses due to missing data. There was insufficient information on about three-quarters of complaints to specify the type of complaint. Clinical specialty was missing from the registers for 75% of registered doctors in the sample.

Construction of study datasets

After cleaning and coding the Victorian and Western Australian data, we combined them in doctor-level and doctor-year level datasets. To construct the doctor-year database, we merged the doctor-level and complaint-level information using a unique practitioner identification number. A doctor registered continuously for 5 years of the study period, for example, would have had five observations in the dataset, one for each year of registration. In addition, a new set of count variables in the dataset indicated the number of complaints, investigations or hearings, and adverse findings accrued by each registered doctor in each year. Complaints missing practitioner identification numbers (< 0.02%) were dropped, as were doctor-years in which the doctor had been registered for fewer than 6 months (< 3%).

Statistical analysis

We calculated counts and proportions to describe the registered doctors and the complaints. To calculate predictors of complaints, we fitted three logistic regression analyses using the doctor-year level dataset. In the first model, the outcome variable was a binary variable indicating whether one or more complaints occurred in the doctor-year; the predictor of interest was a binary variable indicating whether the doctor was an IMG; and the covariates were doctor’s sex and years since qualification, practice location, registration type, year of complaint and state. The model corrected for standard errors for clustering at the doctor level (ie, to account for repeated measures of the same doctors across registration years).

The second model was identical to the first model, except for the outcome variable, which was a binary variable indicating whether one or more adverse findings occurred in the doctor-year. The third model was also identical to the first model, except for the predictor of interest. Instead of specifying a single IMG variable, we created categories specifying 20 different countries of qualification. The named countries were the ones with the most registered doctors; the rest — more than 100 countries, accounting for 1619 doctors — were collapsed into an “other” category.

P < 0.05 was considered statistically significant.

Sensitivity analyses

We tested the sensitivity of results from the three multivariable models that formed our main analyses to an alternative specification of the outcome variables. Instead of treating complaints and adverse findings as binary outcomes, we specified them as count outcomes. The reanalyses were conducted using negative binomial regression and run on both doctor-year level and doctor-level datasets. In the doctor-level analyses, time-varying covariates (years since qualification, registration type, practice location) were set to baseline values.

Results
Characteristics of doctors and complaints

The study sample consisted of 196 646 doctor-years and 39 155 unique doctors, 69% of whom were registered in Victoria and 31% of whom were registered in Western Australia (Box 1). Thirty-seven per cent of doctors had gained their primary medical qualification outside Australia. These IMGs graduated from institutions located in more than 120 different countries.

The boards received 5323 complaints against 3191 doctors during the study period, an incidence of 27 complaints per 1000 doctor-years. Boards investigated or conducted hearings for 30% (1606/5323) of complaints, and an adverse finding was made against the doctor in 373 of these cases.

Multivariable predictors of complaints and adverse findings

IMGs had higher odds of complaints (odds ratio [OR], 1.24; 95% CI, 1.13–1.36; P < 0.001) and adverse disciplinary findings (OR, 1.41; 95% CI, 1.07–1.85; P = 0.01) than Australian-trained doctors.

Disaggregation of the IMG variable into the specific countries of qualification showed wide variation in the odds of complaints against IMGs relative to Australian-trained doctors (Box 2). Doctors who qualified in Nigeria, Egypt, Poland, Russia, Pakistan, the Philippines and India had higher odds of complaints than Australian-trained doctors. Doctors trained in Iran and Sri Lanka also had higher odds of complaints; however, the differences were close to being, but were not actually, significant. Although doctors trained in several countries (Hong Kong, Bangladesh, Germany, Malaysia, New Zealand) appeared to have substantially lower odds of complaints than Australian-trained doctors, the differences were not significant.

The odds ratios from the multivariable models corresponded to annual risks of a complaint ranging from 1.2% among doctors trained in Hong Kong to 8.6% among doctors trained in Nigeria.

Sensitivity analyses

Reconstructing the multivariable analyses as negative binomial models of counts produced very similar results to those reported in our main models, with several minor exceptions. The doctor-level analyses that focused on country of training showed Sri Lanka and Iraq as additional countries associated with significantly higher risk of complaints, and the Philippines was not significant in this model (the online Appendix shows the full set of results from these sensitivity analyses as well as the main analyses).

Discussion

Our study found that IMGs faced 24% higher odds of attracting complaints than non-IMGs, and 41% higher odds of adverse findings. Disaggregation into specific countries of training showed that the overall tendency of IMGs to attract complaints was driven primarily by a significantly higher incidence of complaints among doctors trained in seven countries (Nigeria, Egypt, Poland, Russia, Pakistan, the Philippines, India). IMGs from the 13 other countries examined were no more likely than Australian doctors to attract complaints.

While previous research has consistently shown that certain doctor characteristics, such as male sex,21 are positively associated with complaint risk, the international evidence regarding IMG status as a risk factor is less clear. Two studies10,11 have found no relationship between IMG status and complaint frequency, although one recent analysis from the UK found that IMGs had disproportionately high rates of referral to the National Clinical Assessment Service.22 Other studies have focused on the relationship between IMG status and the outcomes of complaints — specifically, the incidence of adverse findings — and the results from these studies are mixed.12-17 A recent study17 comparing UK-trained doctors with two groups of IMGs (those who qualified elsewhere in the European Union and those who qualified outside the European Union) found that inquiries made to the General Medical Council about both non-UK groups were more likely to attract “higher impact” decisions at each stage of the complaints and disciplinary process.

In finding higher risks among IMGs in Australia, our study extends previous analyses in two directions. First, we profile complaint risk by country of training. A noteworthy limitation of the international research into complaints against IMGs is that they consider IMG status crudely, comparing domestically trained doctors to IMGs en masse or in broad groupings. By this logic, Nigerian-trained and New Zealand-trained doctors working in Australia would both be classified as IMGs. The wide spectrum in risk we identified highlights the perils of considering IMG status in this unitary manner. Similar or lower risks among doctors trained in some foreign countries may attenuate or mask heightened risks among doctors trained in others. It seems plausible that this type of variation underlies the null effect found in most previous studies.

Second, rather than simply focusing on the beginning of the disciplinary process (the occurrence of a complaint), we examined outcomes of the process as well. We found that IMGs also had higher risks of adverse findings. This result is consistent with findings from the analysis of adjudications at successive stages within the UK’s General Medical Council.17 It was not feasible in our study to examine risks of adverse findings by country of training because, with only 373 adverse findings across the study sample as a whole, the analysis was underpowered to reliably detect significant differences.

Why is medical training in certain countries associated with higher risks of complaints? Unfortunately, our findings provide no clear answer. The “at risk” countries we identified share some similar features: English is not the primary language, and all have medical education and health systems that are quite different to Australia’s. However, this explanation is incomplete because the same can be said of several other countries (eg, Bangladesh, China) whose trainees did not exhibit higher risks of attracting complaints. Complainant factors may also play a role, with higher risks of complaints against doctors from some countries possibly reflecting cultural biases. More research is needed to elucidate the reasons for the inter-country differences.

Our study findings should be interpreted in light of the fact that multivariable models estimate the strength and significance of the association between IMG status and complaint risk, not the contribution of IMGs to the overall burden of complaints and adverse findings. Despite disproportionately high complaint rates among doctors trained in certain countries, their contribution to total complaints was small. For example, Nigerian-trained doctors were responsible for only 21 complaints and Polish-trained doctors for only 28 complaints. Consideration of total complaints spotlights India as especially important. Indian-trained doctors had 60% higher odds of attracting complaints than Australian-trained doctors and they accounted for 5% of total complaints lodged during the study period.

Our study had limitations. Complaints to medical boards are merely one marker of quality — they are a kind of sentinel-event surveillance system best suited to flagging problems with care that are both highly visible to patients and which provoke high levels of dissatisfaction.18,19 In addition, we did not have access to information on doctor specialty, complaint type and practice type (eg, sole versus group versus hospital). Inclusion of that information in future research may help shed light on why some IMGs are more complaint-prone than others.

The recent move to a national registration framework should expand opportunities for research in this area. Registration and notification data held by the Australian Health Practitioner Regulation Agency (AHPRA) include specialty and some information on complaint types, cover all states and territories, and capture analogous information on nurses and other types of health professionals. However, because national boards have been in operation for only 2 years, and state-level data from previous years is incomplete and difficult to merge, it will be several years before AHPRA data are ripe for such interrogation.

Findings from this study should provoke and inform discussion about more sophisticated approaches to regulating IMGs. Australia currently has a fast-track system of assessment — the “competent authority pathway” — for medical graduates from five countries (UK, Canada, US, New Zealand, Ireland) in which the quality and similarity of training is regarded as broadly equivalent to domestic programs. (Reassuringly, in our analysis New Zealand and the UK and Ireland had complaint rates that did not differ from those of Australian trainees.) Many other countries operate similar fast- and slow-track systems.

Which countries of training belong in fast-track groups, and on what basis? Should graduates from particular countries be singled out for especially rigorous registration and oversight, including start-up assistance with integration, and ongoing support, mentoring and monitoring? How many tracks are appropriate? By probing the relationship between the country of training and a quality measure, our study casts the spotlight on the logic of existing pathways and highlights the potential value of establishing new pathways. Any moves toward greater regulatory tiering should be evidence-based, both to defend against charges of parochialism and racial profiling and to maximise prospects of delivering real benefits to the health system in terms of improved safety and quality.

1 Characteristics of doctors (n = 39 155) and complaints (n = 5323) in the study sample*

Characteristic

Doctors, no. (%)

Complaints, no. (%)


State

Victoria

27 114 (69%)

4 428 (83%)

Western Australia

12 041 (31%)

895 (17%)

Doctors’ sex

Male

25 587 (65%)

nr

Female

13 555 (35%)

nr

Doctors’ years since qualification

< 10

14 394 (37%)

nr

10–20

11 263 (29%)

nr

21–30

6 715 (17%)

nr

31–40

3 919 (10%)

nr

> 40

2 826 (7%)

nr

Registration type

General

27 406 (70%)

nr

Non-general

11 621 (30%)

nr

Area of need

2 985 (8%)

nr

Specialist

2 743 (7%)

nr

Other

5 893 (15%)

nr

Practice location

Urban

26 778 (81%)

nr

Rural

6 212 (19%)

nr

Country of qualification

Australia

24 542 (63%)

3 728 (70%)

United Kingdom/Ireland

5 129 (13%)

394 (7%)

India

1 871 (5%)

291 (5%)

New Zealand

1 422 (4%)

86 (2%)

South Africa

893 (2%)

93 (2%)

Sri Lanka

689 (2%)

101 (2%)

Germany

311 (1%)

9 (< 1%)

Egypt

292 (1%)

116 (2%)

Iraq

291 (1%)

53 (1%)

China

256 (1%)

23 (< 1%)

Singapore

249 (1%)

32 (1%)

Pakistan

228 (1%)

25 (< 1%)

Malaysia

204 (1%)

8 (< 1%)

Bangladesh

184 (< 1%)

12 (< 1%)

Hong Kong

172 (< 1%)

11 (< 1%)

Philippines

148 (< 1%)

18 (< 1%)

Netherlands

131 (< 1%)

4 (< 1%)

Iran

127 (< 1%)

9 (< 1%)

Nigeria

124 (< 1%)

21 (< 1%)

Russia

120 (< 1%)

38 (1%)

Poland

95 (< 1%)

28 (1%)

Other

1 619 (4%)

220 (4%)

Stage of process

Doctors, no. (%)

Doctors, no. (%)

Complaint

3 191 (8%)

5 323 (100%)

Investigation/hearing

1 171 (3%)

1 606 (30%)

Adverse finding

308 (1%)

373 (7%)


nr = data not reported, as values are not easily interpretable at the complaint level. * For doctor-level variables that may have changed for some doctors over the study period (eg, registration type), we report baseline values. Percentages were calculated using the number of doctors for whom the variable of interest was available as the denominator. Data were missing for < 1% of all variables, except for practice location, which had data missing for 6165 doctors (16%). All countries of qualification other than those specified.

2 Multivariable odds of complaints being made against international medical graduates, by country of training*

Odds ratio (95% CI)

P

Adjusted annual probability of complaint


Australia (reference)

1.00 

2.4%

Nigeria

4.02 (2.38–6.77)

< 0.001

8.6%

Egypt

2.32 (1.77–3.03)

< 0.001

5.2%

Poland

2.28 (1.43–3.61)

< 0.001

5.1%

Russia

2.21 (1.14–4.26)

0.02

5.0%

Iran

1.85 (0.87–3.93)

0.11

4.2%

Pakistan

1.80 (1.09–2.98)

0.02

4.1%

Philippines

1.80 (1.08–3.00)

0.02

4.1%

India

1.61 (1.33–1.95)

< 0.001

3.7%

Sri Lanka

1.33 (0.99–1.78)

0.06

3.1%

Iraq

1.15 (0.70–1.87)

0.59

2.7%

Singapore

1.05 (0.69–1.61)

0.82

2.5%

South Africa

1.06 (0.80–1.42)

0.68

2.5%

United Kingdom/ Ireland

1.00 (0.86–1.17)

0.98

2.4%

Netherlands

0.97 (0.23–4.10)

0.96

2.3%

China

0.90 (0.53–1.50)

0.67

2.1%

New Zealand

0.83 (0.59–1.16)

0.28

2.0%

Malaysia

0.67 (0.28–1.59)

0.36

1.6%

Germany

0.59 (0.26–1.30)

0.19

1.4%

Bangladesh

0.56 (0.24–1.28)

0.17

1.3%

Hong Kong

0.49 (0.18–1.30)

0.15

1.2%


* The model adjusted for doctor sex, doctor practice years, registration type, complaint year, practice location, and state. Not shown is an “other country of training” category, which grouped doctors trained in countries other than those specified (3.2% of all doctor-years). A full set of results for the model is available in the Appendix.
These figures are derived from the multivariable model and indicate doctors’ adjusted probabilities of complaints by country of training, within each doctor-year.

Received 12 April 2012, accepted 20 September 2012

  • Katie Elkin1,2
  • Matthew J Spittal1
  • David M Studdert1,2

  • 1 Melbourne School of Population Health, University of Melbourne, Melbourne, VIC.
  • 2 Melbourne Law School, University of Melbourne, Melbourne, VIC.

Correspondence: d.studdert@unimelb.edu.au

Acknowledgements: 

We thank the Medical Board of Western Australia and the Medical Practitioners Board of Victoria (both now part of the Medical Board of Australia) for providing access to the data used for this study. This study was funded by an Australian Research Council Federation Fellowship (FF0668632) to David Studdert.

Competing interests:

No relevant disclosures.

  • 1. Australian Institute of Health and Welfare. Medical labour force 2006. Canberra: AIHW, 2008. (AIHW Cat. No. HWL 42; National Health Labour Force Series No. 41.) http://www.aihw.gov.au/publication-detail/?id=6442468174 (accessed May 2011).
  • 2. Hawthorne L, Birrell B. Doctor shortages and their impact on the quality of medical care in Australia. People Place 2002; 10: 55-67.
  • 3. Organisation for Economic Co-operation and Development. International migration of health workers. Policy brief, February 2010. http://www.oecd.org/dataoecd/8/1/44783473.pdf (accessed Oct 2011).
  • 4. Gorman DF, Brooks PM. On solutions to the shortage of doctors in Australia and New Zealand. Med J Aust 2009; 190: 152-156. <MJA full text>
  • 5. Hawthorne L, Hawthorne G, Crotty B. Final report: the registration and training status of overseas trained doctors in Australia. Melbourne: University of Melbourne, 2007.
  • 6. Wijesinha S. Australians see doctor death in every overseas-trained doctor. The Age 2005; 14 Jun. http://www.theage.com.au/news/Opinion/Australians-see-Dr-Death-in-every-overseastrained-doctor/2005/06/13/1118645745078.html (accessed Sep 2012).
  • 7. Dunbar J, Reddy P, May S. Deadly healthcare. Bowen Hills, Queensland: Australian Academic Press, 2011.
  • 8. Norcini JJ, Boulet JR, Dauphinee WD, et al. Evaluating the quality of care provided by graduates of international medical schools. Health Aff (Millwood) 2010; 29: 1461-1468.
  • 9. Ko DT, Austin PC, Chan BTB, Tu JV. Quality of care of international and Canadian medical graduates in acute myocardial infarction. Arch Intern Med 2005; 165: 458-463.
  • 10. Cunningham W, Crump R, Tomlin A. The characteristics of doctors receiving medical complaints: a cross-sectional survey of doctors in New Zealand. N Z Med J 2003; 116: U625.
  • 11. Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA 2007; 298: 993-1000.
  • 12. Morrison J, Wickersham P. Physicians disciplined by a state medical board. JAMA 1998; 279: 1889-1893.
  • 13. Morrison J, Morrison T. Psychiatrists disciplined by a state medical board. Am J Psychiatry 2001; 158: 474-478.
  • 14. Khaliq AA, Dimassi H, Huang CY, et al. Disciplinary action against physicians: who is likely to get disciplined? Am J Med 2005; 118: 773-777.
  • 15. Kohatsu ND, Gould D, Ross LK, Fox PJ. Characteristics associated with physician discipline: a case–control study. Arch Intern Med 2004; 164: 653-658.
  • 16. Clay SW, Conatser R. Characteristics of physicians disciplined by the State Medical Board of Ohio. J Am Osteopath Assoc 2003; 103: 81-88.
  • 17. Humphrey C, Hickman S, Gulliford MC. Place of medical qualification and outcomes of UK General Medical Council “fitness to practise” process: cohort study. BMJ 2011; 342: d1817.
  • 18. Hickson GB, Federspiel CF, Pichert JW, et al. Patient complaints and malpractice risk. JAMA 2002; 287: 2951-2957.
  • 19. Bismark MM, Studdert DM. Realising the research power of complaints data. N Z Med J 2010; 123: 12-17.
  • 20. Stevenson DG. Nursing home complaints and quality of care: a national view. Med Care Res Rev 2006; 63: 347-368.
  • 21. Firth-Cozens J. Doctors with difficulties: why so few women? Postgrad Med J 2008; 84: 318-320.
  • 22. National Clinical Assessment Service. Concerns about professional practice and associations with age, gender, place of qualification and ethnicity – 2009/10 data. London: NCAS, 2010.

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Responses are now closed for this article.