Selecting medical students: we need to assess more than academic excellence

Medical schools require selection processes that reflect the type of doctor they aim to produce

Selection for medical school is based on the applicant’s academic record, aptitude testing, and assessment of their personal attributes. The indicator of subsequent performance best supported by evidence is prior academic attainment,1,2 with the evidence coming mostly from exam performance at medical school and in postgraduate specialties. Rationales for employing further selection criteria have included alignment with professional body guidance,3 better discrimination between equally qualified applicants, and recognition that becoming a good doctor requires qualities beyond academic excellence.4 In light of current practice, which selection criteria are necessary (ie, provide a minimum required threshold) and which may be sufficient, singly or in combination, for selecting medical students?

In this issue of the MJA, Shulruf and his co-authors5 argue that completion of medical school is a better proxy measure of competence than exam performance per se because it better reflects real world outcomes and therefore, presumably, future patient safety. The authors analysed this outcome at five Australian and New Zealand universities, and found that prior educational attainment was the best predictor, aptitude testing (the Undergraduate Medicine and Health Sciences Admission Test [UMAT]) and interview not markedly improving discrimination between students who succeeded or failed. Based on these results, they propose that simple cut-scores would be a sufficient criterion for entry selection; for three universities this would consist of prior academic attainment only, for two a combination of attainment and UMAT. Interviews would be redundant in all cases.

For a selector dealing with many more applicants than available places, this proposal may be beguiling. However, it effectively implies that no criterion is necessary for entry to medical school other than evidence of previous academic excellence, as this predicts competence at graduation, and therefore future patient safety. This idea, however, conflates two distinct purposes of selection: establishing that a candidate is suitable, and distinguishing between competing suitable candidates.

Firstly, what are the necessary minimum thresholds for anyone wishing to become a doctor? Some candidates will be unsuitable because they lack the cognitive ability to acquire essential competencies (eg, recognising pulmonary embolism, deciding about appropriate management). Assessing the candidates’ academic records and their cognitive aptitude is therefore required. Could a single threshold be optimal? In the article by Shulruf and colleagues, the derived optimal cut-scores for predicting completion varied substantially between schools — for example, for GPA/ATAR they ranged from 88.1 to 98.6 — and therefore did not provide an absolute threshold for assessing cognitive ability. An applicant might also be unsuitable because they lack essential behavioural attributes,6,7 perhaps to such a degree that the deficit is regarded as irremediable; for instance, serious or repeated criminal or antisocial behaviour, difficulty in perspective taking, black-and-white thinking, or lack of personal insight and reflection. Assessing both academic ability and personal qualities are accordingly essential.

Secondly, let us consider how we rank or discriminate between competing suitable candidates, all of whom reach the necessary minimum thresholds. At this stage, one appropriately takes into account the specific mission of the particular institution: one school may be geared towards training the academic and clinical researchers of the future, another aims for practitioners for remote and rural medicine. The first program may well include considerable academic challenge, so that discriminating according to past academic record or current aptitude test performance could be the best approach. In the second example, other attributes may be critical — upbringing or schooling in rural areas, a strong vocational motivation to serve underprivileged communities, a high degree of self-reliance — that might be better assessed in an interview or situated judgment test. In both cases, the institutions require evidence-based, defensible selection criteria, some of which may be based on their own data, while others may benefit from multi-institutional data and analysis, such as that provided by the study by Shulruf and colleagues or the Australian Medical Schools Outcomes Database (MSOD).8 Sufficient selection evidence (once suitability is established) could then be drawn from predictors of exam performance or predictors of practice outcomes after qualification, such as remote or rural practice location, or successful clinical research.

There is one last matter to consider: other than academic excellence, what factors are responsible for medical students failing to complete their programs? Research has found that personal, social and health (especially mental health) factors are responsible in most cases, with only a minority attributable to academic failure.9,10 The response to these findings may include better support during training or applying better selection tools, but one should also consider that the overall attrition rate in medical schools is very low (2.6% in one recent major study2) and may, in fact, be acceptable.