Multiple mini interview performance of repeat applicants to medical school admission

To the Editor: As the use of the multiple mini interview (MMI) for selecting applicants for admission to health professional programs increases, so does the number of coaching classes assumed to help improve performance on the MMI.1 Moreover, applicants applying to several professional programs have multiple opportunities to gain experience in a high-stakes environment. These realities usher concerns about the influence of “practice effect” on subsequent MMI performance.2 While short-term prior access to MMI questions did not influence applicants' performance in a Canadian medical school,3 in Australia, MMI participation in the year after the original attempt improved scores on stations (short, structured interviews to assess personal qualities such as communication, professionalism, ethics skills) that were the same or similar to original MMI stations.2

At the University of Manitoba, Canada, about 25% of 1151 unique applicants to the Faculty of Medicine's Undergraduate Medical Education program from 2008 to 2012 were those unsuccessful at gaining medical school admission in previous years. While 856 applicants (74.4%) appeared once for the MMI, 208 (18.1%) appeared twice, 64 (5.6%) appeared thrice, 16 (1.4%) appeared four times and seven (0.6%) appeared five times. In the faculty's 12-station MMI,4 88 unique question stems or station scenarios were used in the 5-year period. The number of questions used once, twice, thrice, four and six times was 46, 18, 17, 5 and 2, respectively, with at least a 1-year gap between original and repeat use, as recommended.2

To assess whether the scores of those who repeated the MMI were higher, applicants' average scores and within-year z scores were compared by calendar year, number and order of attempts and then analysed by multiple linear regression.5

There was a decreasing trend in raw MMI scores for all applicants between 2008 and 2012. The mean MMI score at first attempt of one-time applicants was 4.66 (SD, 0.65) on a scale of 1 to 7, while that at first attempt of 295 repeat applicants was 4.25 (SD, 0.60) (P < 0.001). The mean MMI score of all 707 attempts by the 295 repeat applicants improved slightly to 4.38 (SD, 0.64). After adjusting for age, sex, and calendar year, regression parameter estimates associated with 2nd, 3rd and 4th attempts, but not the 5th, gradually and significantly increased relative to the 1st attempt. Yet, only 9.1% of the variability in within-year z-transformed MMI scores was explained by the selected model.

It is unlikely that repeat attempts at MMI alone systematically and considerably benefited applicants. Other unmeasured and unexplored factors, such as individuals' personal growth and maturity, may play a role in positively influencing MMI scores. Our study did not account for the effect of coaching that a substantial number of Australian medical school applicants reportedly attend.1,2 Despite some limitations in our study design and statistical methods, our results currently do not lend support to the concern that repeated MMI performance results in improved MMI scores.