Connect
MJA
MJA

Removing the interview for medical school selection is associated with gender bias among enrolled students

David Wilkinson, Mavourneen G Casey and Diann S Eley
Med J Aust 2014; 200 (2): 96-99. || doi: 10.5694/mja13.10103
Published online: 3 February 2014

Abstract

Objective: To report, and determine reasons for, a change in the gender ratio observed among enrolled medical students after removal of the interview from the selection process.

Design, setting and participants: Cross-sectional study of 4051 students admitted to the medical program at the University of Queensland between 2004 and 2012. Students are enrolled either directly as graduates or via a school-leaver pathway.

Main outcome measures: Change in proportions of male and female students over time, and gender-specific scores in the three sections of the GAMSAT (Graduate Medical School Admissions Test).

Results: Between 2004 and 2008 (when an interview was part of the selection process), 891 enrolled students (51.4%) were male, whereas between 2009 and 2012 (no interview), 1134 (57.7%; P < 0.001) were male. This change in gender ratio was limited to domestic direct graduate-entry students, and the male proportion in this group rose from 50.9% (705 students) before the interview was removed to 64.0% (514 students; P < 0.001) after removal of the interview (reaching 73.8% in 2012). Between 2004 and 2012, male students consistently performed better than female students on GAMSAT section III (mean score, 71.5 v 68.5; P < 0.001).

Conclusion: The proportion of males enrolled in the medical program at this university increased markedly after removal of the interview from the selection process. This change is limited to domestic direct graduate-entry students, and seems to be due to higher scores by male students in section III of the GAMSAT. The interview may play an important role in ensuring gender equity in selection, and medical schools should carefully monitor the consequences of changes to selection policy.

Selection processes for entry into medical school are continually under scrutiny, not only for predictive validity, but also for equity and fairness.1-3 The importance of equitable selection criteria that can identify candidates who will succeed academically and negotiate the transition into professional practice is widely acknowledged.3,4 Simultaneously, broadening the admissions base is an important part of social accountability, and selection processes at some schools include mechanisms to select students based on background, social status and personal qualities, as well as academic performance.5,6 Mechanisms employed to do this include quotas, personal statements, portfolios and interviews.1,4,7

Internationally, debate continues about who to admit to medical study, and how to admit them.3 Equity issues identified include gender, age, socioeconomic status, rurality and Indigenous status.8,9 Prior academic performance has the highest predictive validity for performance in medical school but has itself been shown to be influenced by gender, educational background and social class.1,3,9,10 Despite their widespread use, evidence suggests that interviews have poor predictive validity and limited practicality and are subject to bias, including gender bias.11-14

Australian medical schools use various combinations of factors in making selection decisions, including prior academic performance (at high school or university), performance on standardised tests including UMAT (Undergraduate Medicine and Health Sciences Admissions Test) or GAMSAT (Graduate Medical School Admissions Test), a written statement or portfolio and an interview.15,16 All medical schools admitting graduates make use of the GAMSAT.15 Although the criterion validity of the GAMSAT is continually reviewed, questions remain about its predictive value and gender bias.15-18 Reports have shown a modest but statistically significant gender bias in favour of male students.17

In 2009, after analysis of the format, validity and utility of the interview, this component of the selection process was discontinued at the University of Queensland (UQ).18 Here, we report a consequent change in the gender ratio among enrolled students and analyse the reasons for this shift.

Methods
Setting and subcohorts

The medical degree at UQ is a 4-year graduate-entry program, with an intake in 2012 of 543 students. The full cohort is made up of distinct subcohorts:

  • domestic students: about 300 places are available each year, as determined by the federal government; and

  • international students: about 120 places for students who spend all 4 years in Australia, and (in 2012) about 85 places for students from the United States who spend the first 2 years of their program in Australia and the second 2 years in the US.

In addition, there are two distinct entry pathways:

  • graduate-entry pathway (domestic): about half of all domestic places are available to students with a prior degree who apply for direct entry into the MB BS program; and

  • school leaver pathway, which is also called a dual degree (domestic): about half of all domestic places are available to students provisionally accepted directly from high school who first study a non-medical undergraduate degree before commencing their MB BS.

Admissions criteria

Admissions criteria are found on the UQ website (http://www.uq.edu.au). Before 2009, selection to the domestic graduate-entry pathway was based on exceeding a defined GPA (grade point average) in a prior degree, and then on ranking based on a composite score of performance in the GAMSAT and the interview. Entry to the domestic school leaver pathway was based on attaining the highest level of high school score (called an Overall Position 1 or equivalent), and then ranking based on the UMAT total score. (There has never been an interview for this subcohort.) From 2009 onwards, the interview was not used for graduate entry, but other criteria remained the same. International students are selected in a similar manner, but may take the North American MCAT (Medical College Admission Test) instead of the GAMSAT and the ISAT (International Student Admissions Test) instead of the UMAT. There have never been interviews for international students.

Graduate Medical School Admissions Test

The GAMSAT has been used since 1996 in Australia and is a written aptitude test developed and administered by the Australian Council for Educational Research (ACER) on behalf of the consortium of graduate-entry medical schools (GAMSAT Consortium Universities).19,20 The GAMSAT is divided into three sections: section I, reasoning in humanities and social sciences; section II, written communication; and section III, reasoning in biological and physical sciences. The GAMSAT overall score is computed as a weighted average of the three section scores in the ratios 1:1:2, respectively.19 Each university determines its own cut-off value, and at UQ (at the time of this analysis) a minimum score of 50 in each of the three sections of the GAMSAT test was required to be eligible for entry, with ranking done on overall score and then by GPA within the GAMSAT score band.20

Student cohorts

We studied nine consecutive cohorts (2004–2012) comprising 4051 enrolments in the medical program. We examined student characteristics and GAMSAT scores over this period, and any changes from the period when an interview was in place (2004–2008) to the period after the interview was discontinued (2009–2012). Student characteristics studied included gender, entry pathway (graduate or school leaver) and international or domestic status.

Statistical analysis

Statistical analysis was performed using SPSS version 21 (SPSS Inc). Descriptive statistics were calculated and Pearson χ2 tests performed to test for differences in gender proportion for all entry years combined, for each year and between the interview and no-interview periods. Results are reported with 95% CI, and a P value < 0.05 is regarded as significant. Preliminary Pearson correlations were run to determine the associations between GAMSAT sections (I–III) and the overall score (weighted average) as well as an unweighted mean score. A two-way, multivariate analysis of variance (ANOVA) was conducted to test for gender differences in GAMSAT section scores and to compare differences between gender over the years of the study period (Bonferroni-adjusted P = 0.017). Gender differences between the interview and no-interview periods in GAMSAT overall scores were examined using two-way ANOVA. Mean scores with 95% CI, standard errors, F, P and effect sizes are reported. Preliminary analyses were performed to test for violations of the assumptions of normality, linearity and homoscedasticity; where necessary, significance levels were reported at a more stringent level of P < 0.01.21

Ethics approval

Ethics approval was provided by the UQ Behavioural and Social Sciences Ethical Review Committee. No funding was sought for this study.

Results
Changing characteristics of subcohorts of enrolled students

In all, 4051 students were admitted to study medicine between 2004 and 2012 (3698 students at the School of Medicine and 353 school leavers provisionally accepted into medicine but still completing their first degree) (Box 1). The annual intake into the School of Medicine increased substantially between 2004 (307 students) and 2012 (543 students). The median age of enrolled students was 23 years (interquartile range, 3 years), with no observed differences in mean age between the interview and no-interview periods (24.4 years and 24.5 years, respectively). The proportion of international students increased from 9.8% (30 students) in 2004 to 44.0% (239 students) in 2012. The proportion of domestic students enrolled as direct graduates fell from 100% in 2004–2007 to 71.1% in 2008–2012 (and reached 40.1% in 2012).

Changes in gender ratio

Between 2004 and 2008, 891 enrolled students (51.4%) were male, whereas between 2009 and 2012, 1134 (57.7%; χ2 = 15.01; df = 1; P < 0.001) were male. There was a notable shift in the gender ratio (in favour of males) in 2009, when the proportion of male students jumped from 52.2% in 2008 to 57.7%. This proportion has remained stable within 1% since 2009 (Box 1). In the international subcohort, the proportion of enrolled males has been consistently higher than that of females (by about 5%–10%) across the entire study period, especially since 2007, when the number of international students increased, but this difference is not statistically significant (χ2 = 2.41; df = 8; P = 0.97). There was no significant difference in gender ratio within the international student subcohort between the interview and no-interview periods (χ2 = 0.59; df = 1; P = 0.45). As shown in Box 1, the gender ratio for the domestic school leaver pathway between 2008 and 2012 is almost balanced. However, for the domestic direct graduate-entry pathway, the male proportion rose from 50.9% in 2004–2008 to 64.0% in 2009–2012 (χ2= 35.19; df = 1; P < 0.001). In 2011 the proportion of male students enrolled through the domestic direct graduate-entry pathway was 73.1%, and in 2012 it was 73.8% (Box 2).

GAMSAT scores by gender

Of the 2187 domestic students enrolled by the direct graduate pathway, 2047 GAMSAT scores were available (special admissions were excluded). Mean GAMSAT section scores are reported in Box 3, by gender (for 1151 male and 896 female students). The difference in mean scores in sections I and II between male and female students is small (less than 1 unit), while the difference in mean scores for section III is more than 3 units (71.5 for males v 68.5 for females). The Pearson correlation calculation showed that section III is more strongly correlated with GAMSAT overall scores (r[2045] = 0.85; P < 0.001) than sections I–II (r[2045] = 0.48 for section I, and 0.31 for section II; P < 0.001). By comparison, the correlations between sections (I–III) and an unweighted mean score were closer (r[2045] = 0.64 for section I, 0.57 for section II and 0.62 for section III; P < 0.001).

A two-way, between-group, multivariate ANOVA of GAMSAT section scores, applying Bonferroni adjustments, showed significant gender differences in section III (Box 3). The test showed a significant difference between male and female students on the combined GAMSAT sections I, II and III (F3,2027 = 25.18; P < 0.001; Wilk l = 0.96; partial eta-squared = 0.036), but when the section scores were considered separately, the only difference in gender was in section III (P < 0.001). There was no significant interaction between gender and year of entry on the combined three sections (P = 0.12) or on any of the sections when considered separately (P = 0.19 for section I, P = 0.16 for section II and P = 0.26 for section III), indicating that the influence of gender on differences in scores remained consistent across all years studied. A two-way ANOVA confirmed a significant difference between gender in GAMSAT overall scores (F1,2043 = 34.03; P < 0.001; partial eta = 0.016) but no difference in overall scores by gender between the interview and no-interview periods (P = 0.55).

Discussion

Our analysis shows that the proportion of male students enrolled in the medical program at UQ has increased substantially in recent years. This change is isolated to the domestic direct graduate-entry subcohort of students, and the change started in the first year that the interview was not included in the selection process. The increased proportion of males enrolled overall has persisted since the change in selection practice, and the proportion among domestic direct-entry graduates has increased to 73.8% in 2012.

Our observations are in stark contrast to the general trend in Australia where between 2009 and 2012, 51.7% of all enrolments in Australian medical schools were female students.22 Further, using data supplied by ACER, we found no evidence of an increased proportion of males sitting the GAMSAT or applying for entry to medical school between 2007 and 2011 (Appendix 1). Indeed, among the 12 graduate-entry medical programs, between 2007 and 2011, in only two cases was the proportion of males applying higher than 50% (and the highest of these was 54.1%) (Appendix 2).

The reason for the shift at UQ seems to be the influence of higher scores achieved by male applicants in section III of the GAMSAT. Higher scores on section III have been shown to be associated with higher GAMSAT overall scores, on which candidates are ultimately ranked for selection to medical school.17,20 The results from our analysis of GAMSAT scores are consistent with prior reports of a significant (but relatively small) gender effect favouring males on both section III and overall scores.17 We also reported strong correlations between section III and the overall score but when a mean score (using unweighted sections) was calculated for the UQ cohort, this effect was noticeably moderated.

Alternatively, the strong influence of academic background on GAMSAT performance is well documented.9,16,17 For example, graduates from disciplines such as engineering, where students are predominantly male, perform better on section III.17 While it is important to acknowledge the substantial effect of prior learning, the academic backgrounds of GAMSAT candidates have remained relatively stable over the study period.16 Further, the academic selection criteria at UQ have not changed. The one factor that has changed has been the discontinuation of the admissions interview at UQ.

Our findings have significant policy implications. Currently, all GAMSAT Consortium medical schools in Australia rank applicants using the weighted overall score calculated by ACER. The exception is the University of Melbourne, which uses an average (unweighted) GAMSAT score.20 GAMSAT section III focuses on reasoning in biological and physical sciences, and we find it difficult to rationalise the continued weighting of scores in this section if this practice leads to significant gender bias, as it seems to have done here. Our view is that cohorts of admitted medical students should be representative of the communities from which they are drawn, and which they will later serve.

There is consensus that selection processes aimed at broadening access to medical school have not been extensively studied.3 There are limited studies focusing on gender and the interview process; however, the literature does indicate a shift in attitudes from an earlier bias favouring males.13,14 We were not able to present data on gender differences in interview scores at UQ, but our study does lend support to the more recent studies suggesting that personal interviews favour female applicants.14 Apparently, without the moderating effect of the interview, gender bias in GAMSAT performance has led to a marked increase in the proportion of male medical students enrolled at UQ.

How to best select medical students remains challenging and contentious. Like other medical schools, we use quotas for specific groups, such as Indigenous students, those with a rural background and those following different pathways. We are now considering how best to ensure a balanced gender ratio among enrolled students in our direct-entry graduate pathway and recommend that schools monitor changes to selection policy and practice carefully.

1 Proportions of male and female students, by subcohort, in each year between 2004 and 2012

Domestic students (n = 2626)


International students (n = 1072)


All students (n = 3698)*


Graduate students (n = 2187)


School leaver pathway group§ (n = 439)


School of Medicine entry year

Total

Female

Male

Female

Male

Female

Male

Female

Male


2004

307

156 (50.8%)

151 (49.2%)

145 (52.3%)

132 (47.7%)

11 (36.7%)

19 (63.3%)

2005

304

146 (48.0%)

158 (52.0%)

118 (48.0%)

128 (52.0%)

28 (48.3%)

30 (51.7%)

2006

313

153 (48.9%)

160 (51.1%)

126 (48.5%)

134 (51.5%)

27 (50.9%)

26 (49.1%)

2007

387

186 (48.1%)

201 (51.9%)

157 (48.8%)

165 (51.2%)

29 (44.6%)

36 (55.4%)

2008

423

202 (47.8%)

221 (52.2%)

133 (47.7%)

146 (52.3%)

12 (48.0%)

13 (52.0%)

57 (47.9%)

62 (52.1%)

2009

454

192 (42.3%)

262 (57.7%)

116 (41.1%)

166 (58.9%)

9 (42.9%)

12 (57.1%)

67 (44.4%)

84 (55.6%)

2010

500

216 (43.2%)

284 (56.8%)

99 (40.7%)

144 (59.3%)

34 (50.7%)

33 (49.3%)

83 (43.7%)

107 (56.3%)

2011

467

193 (41.3%)

274 (58.7%)

42 (26.9%)

114 (73.1%)

76 (52.8%)

68 (47.2%)

75 (44.9%)

92 (55.1%)

2012

543

229 (42.2%)

314 (57.8%)

32 (26.2%)

90 (73.8%)

90 (49.5%)

92 (50.5%)

107 (44.8%)

132 (55.2%)

Total medical students

3698

1673 (45.2%)

2025 (54.8%)

968 (44.3%)

1219 (55.7%)

221 (50.3%)

218 (49.7%)

484 (45.1%)

588 (54.9%)

School leaver pathway group (studying first degree)

167 (47.3%)

186 (52.7%)


* χ2 = 16.18; df = 8; P = 0.04. χ2 = 2.41; df = 8; P = 0.97. χ2 = 51.95; df = 8; P < 0.001. § χ2 = 0.929; df = 4; P = 0.92. School leavers with guaranteed entry into medicine but still completing their first non-medical degree at the University of Queensland at the time of analysis and not yet commenced medical course. These students are not included in the column totals.

2 Proportions of male and female medical students enrolled through the direct graduate-entry pathway

3 Mean GAMSAT scores by gender for 2047 medical students enrolled through the direct graduate-entry pathway in 2004–2012

Mean (SEM; 95% CI)


GAMSAT score

Female students

Male students

F (df)

P*

Partial eta-squared


Section I

63.32  (0.21; 62.92–63.73)

62.76  (0.16; 62.44–63.07)

4.62  (1,2029)

0.03

0.002

Section II

64.27  (0.27; 63.73–64.80)

63.59  (0.21; 63.17–64.00)

3.84  (1,2029)

0.05

0.002

Section III

68.45  (0.30; 67.87–69.04)

71.53  (0.23; 71.07–71.98)

65.42  (1,2029)

< 0.01

0.031

Overall score

66.07  (0.17; 65.74–66.41)

67.35  (0.14; 67.08–67.62)

34.03  (1,2043)

< 0.01

0.016


GAMSAT = Graduate Medical School Admissions Test. * Bonferroni-adjusted level = 0.017. Overall score = (1 × section I score + 1 × section II score + 2 × section III score)/4.

Received 21 January 2013, accepted 3 July 2013

  • David Wilkinson1
  • Mavourneen G Casey2
  • Diann S Eley2

  • 1 Macquarie University, Sydney, NSW.
  • 2 School of Medicine, University of Queensland, Brisbane, QLD.

Correspondence: m.casey3@uq.edu.au

Acknowledgements: 

We would like to thank Michelle Doughan at UQ Central, Management Information Systems, for assistance with data extraction from UQ student records.

Competing interests:

No relevant disclosures.

  • 1. Lumsden MA, Bore M, Millar K, et al. Assessment of personal qualities in relation to admission to medical school. Med Educ 2005; 39: 258-265.
  • 2. Bore M, Munro D, Powis D. A comprehensive model for the selection of medical students. Med Teach 2009; 31: 1066-1072.
  • 3. Prideaux D, Roberts C, Eva K, et al. Assessment for selection for the health care professions and specialty training: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011; 33: 215-223.
  • 4. McManus C, Powis D. Testing medical school selection tests. Med J Aust 2007; 186: 118-119. <MJA full text>
  • 5. James D, Yates J, Nicholson S. Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in 2006: cohort study. BMJ 2010; 340: c478. doi: 10.1136/bmj.c478.
  • 6. Hughes P. Can we improve on how we select medical students? J R Soc Med 2002; 95: 18-22.
  • 7. Story M, Mercer A. Selection of medical students: an Australian perspective. Intern Med J 2005; 35: 647-649.
  • 8. Wilkinson D, Birks J, Davies L, et al. Preliminary evidence from Queensland that rural clinical schools have a positive impact on rural intern choices. Rural Remote Health 2004; 4: 340.
  • 9. Blackman I, Darmawan I. Graduate-entry medical student variables that predict academic and clinical achievement. Int Educ J 2004; 4: 30-41.
  • 10. Ferguson E, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. BMJ 2002; 324: 952-957.
  • 11. Goho J, Blackman A. The effectiveness of academic admission interviews: an exploratory meta-analysis. Med Teach 2006; 28: 335-340.
  • 12. Groves MA, Gordon J, Ryan G. Entry tests for graduate medical programs: is it time to re-think? Med J Aust 2007; 186: 120-123. <MJA full text>
  • 13. Marquart JA, Franco KN, Carroll BT. The influence of applicants’ gender on medical school interviews. Acad Med 1990; 65: 410-411.
  • 14. Salvatori P. Reliability and validity of admissions tools used to select students for the health professions. Adv Health Sci Educ Theory Pract 2001; 6: 159-175.
  • 15. Coates H. Establishing the criterion validity of the Graduate Medical School Admissions Test (GAMSAT). Med Educ 2008; 42: 999-1006.
  • 16. Wilson IG, Roberts C, Flynn EM, Griffin B. Only the best: medical student selection in Australia. Med J Aust 2012; 196: 357. <MJA full text>
  • 17. Pywell S, Hunt M, Edwards J, et al. Report on Graduate Australian Medical School Admissions Test 2012. Camberwell, Vic: Australian Council for Educational Research (ACER), 2012.
  • 18. Wilkinson D, Zhang J, Byrne GJ, et al. Medical school selection criteria and the prediction of academic performance. Med J Aust 2008; 188: 349-354. <MJA full text>
  • 19. Australian Council for Educational Research (ACER). Graduate Australian Medical School Admissions Test (GAMSAT) Information Booklet 2012. http://gamsat.acer.edu.au/gamsat-australia/info-booklet (accessed Dec 2012).
  • 20. Gradauate Entry Medical Schools Admissions System (GEMSAS). Graduate medical school admisions guide 2014-2013. http://www.gemsas.edu.au/wp-content/uploads/2013/05/2014-Medicine-GEMSAS-Admissions-Guide-v1-5.pdf (accessed May 2013).
  • 21. Parra-Frutos I. The behaviour of the modified Levene’s test when data are not normally distributed. Comput Stat 2009; 24: 671-693.
  • 22. Medical Deans Australia and New Zealand (MDANZ). Annual tables. Medical student statistics. http://www.medicaldeans.org.au/statistics/annualtables (accessed Dec 2012).

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

access_time 05:28, 5 February 2014

<p>&nbsp;</p><p>The assertion of the authors that the GAMSAT scores and interview scores are independent in the pre-2009 UQ admission process is not strictly true. The admissions process at the UQ Medical School pre-2009 operated by filtering the candidates on the basis of GAMSAT scores on two criteria: that a candidates sectional scores were &gt;50 in each section and; the total score, defined as an unweighted sum of the sectional scores was above a cut off value set by the admissions officer to give a pre-defined number of interview candidates. There were some variations to this scheme applied in the case of ATSI candidates. The final score was a simple sum of GAMSAT plus interview scores. The candidate ranking on final scores was used to make offers, offer scholarships, etc.</p><p>I note that ACER now quotes the GAMSAT score as a weighted sum of sectional scores. The authors make no comment on the different construction of GAMSAT scores and indeed make no comment that the method of deriving GAMSAT scores has changed over the years. One may be forgiven for wondering if the change in gender ratios at UQ may in part be due to the construction of GAMSAT scores. The ACER handbook on the GAMSAT test notes that Melbourne Uni retains the old scheme for the construction of a GAMSAT score - perhaps with good reason.</p><p>&nbsp;</p>

Competing Interests:

access_time 10:12, 24 May 2015
Glen Kerr Laslett

My experience (buttressed by research) is that interview processes are generally less reliable indicators of future performance then testing. They may be useful however for assessing cultural fit. It would be interesting to review the longitudinal performance of the UQ graduate cohorts to determine whether there was any performance benefit (or detriment) arising from sole reliance on the GAMSAT (and in particular Section III).

Competing Interests: No relevant disclosures

Mr Glen Kerr Laslett
University of Sydne

Responses are now closed for this article.