Reducing the impact of coaching on selection into medicine

Coaching to enter medical school first attracted research attention in 2008.1 Results indicated that just over half of those shortlisted for interview had received commercial coaching for the Undergraduate Medical and Health Sciences Admissions Test (UMAT), which is used to select interview candidates. Evidence indicates that students who had received UMAT coaching subsequently show significantly poorer academic performance throughout their medical degree compared with those who had not been coached.2 This suggests that UMAT scores achieved after coaching may not represent true ability to do medicine, or that students who rely on coaching cope less well in academic environments where coaching is not appropriate.

A study of Year 12 high school students in New South Wales and Victoria showed that coaching had no impact on the UMAT sections that assess problem solving or understanding people, but coached students had slightly higher scores on the non-verbal test of logical reasoning.3 Similar results were obtained in a New Zealand study, which involved students who had been coached by the MedEntry company.4 Even though these students believed that their UMAT performance would be improved, this belief was misplaced as their UMAT results were no better than those of uncoached students.

Last year, the University of Western Sydney altered its metric for shortlisting applicants for their Multiple Mini Interview. Subsequently, only 35.5% of those invited to interview had engaged in commercial UMAT coaching, a significant decrease from the 51.4% of interviewees who were coached in 20081 (χ² = 7.43; P = 0.003). There was no statistical difference between coached (n = 122) and uncoached (n = 222) interviewees on any of the three UMAT scores (P = 0.891, 0.885 and 0.945 for UMAT Sections 1, 2 and 3, respectively) or the Multiple Mini Interview scores (P = 0.352). Thus, the coached group were no more likely to gain entrance to the medical program. However, the coached group had higher academic university entrance ranking scores (ATAR) than their uncoached peers (mean ATAR = 98.76 v 98.01; t = 2.99; P = 0.003). Applicants who are likely to be shortlisted for interviews and yet feel the need for UMAT coaching may be less confident in their own ability or more susceptible to industry advertising — but the time and money spent on commercial coaching appears to be misdirected.