Only the best: medical student selection in Australia

Over the past two decades there has been a significant change in the way Australian medical schools select their students. Where once a school leaver’s matriculation score was the predominant criterion,1 there is now a range of selection procedures for entry into school-leaver, graduate-entry and mixed-entry medical school programs. The change in selection procedures has in part been driven by a desire to assess broader suitability than just academic performance, and the need for medical schools to be socially accountable and reduce discrimination in selection procedures.2

We provide an overview of medical student selection in Australia, including the aims and principles of selection, the research into selection methods and limitations of that research.

Selection strategies broadly consider educational achievement, aptitude for future study and desirable personal characteristics, typically those of entry-level professionalism, such as empathy and communication skills. The selection strategies of individual medical schools attempt to balance twin aims in identifying applicants — the greatest aptitude for successfully completing a medical course and the characteristics required to be competent medical practitioners.

To achieve this, selection processes need to have clear aims. They must be valid, reliable and unbiased, and as evidence-based as possible. They also need to be closely matched to the chosen selection criteria, defensible and subject to ongoing quality assurance. Selection should also be related to other aspects of medical education, and aligned with the school’s curriculum and its graduate outcomes. Above all, the processes must be transparent and trusted by all stakeholders.

Medical schools generally use a combination of academic and non-academic measures to assess suitability. Academic measures consist of prior academic performance and written problem-solving tests, while non-academic measures include interviews and written work to assess applicants’ values and personal characteristics. The wide range of tools currently in use in Australia, and their benefits and limitations are summarised in Box 1.

The World Health Organization regards medical schools as having an obligation to direct their education towards the health needs of the populations they serve.2 To meet these social commitments, universities have adjusted their selection processes to increase applications from candidates who could serve special-needs communities. A number of Australian medical schools have inbuilt systems to encourage the entry of students from areas of workforce need, most notably from rural regions, based on the premise that they are more likely to return to these areas to practise.14 In addition, federal government policy has promoted both entry of rural and regional students to medical training and, through quotas and scholarships, the requirement to work in areas of need (Medical Rural Bonded Scholarship Scheme <www.health.gov.au/mrbscholarships> and Bonded Medical Places Scheme <www.health.gov.au/bmpscheme>).

There are also special access pathways and support structures for socially disadvantaged applicants. In Australia, this mostly applies to Indigenous students, but schemes also exist to recruit students from refugee and other disadvantaged backgrounds.

The way in which the final ranking of applicants is developed can provide a clear message to stakeholders of the relative importance of academic ability, aptitude, professional attitudes and social accountability. The approach to ranking varies significantly between medical schools, with policy being driven by a complex interplay of local, regional and national priorities.15 Some medical schools use a proportion of all scores in developing a final ranking, while others use certain measures as cut-off scores only and rely more on interviews to rank students.

There is substantial international interest in demonstrating the robustness and equity of medical school selection processes. Recent publications have reported on the utility, reliability and validity of the various test formats, particularly those for assessing non-academic criteria.12,16 - 19 Most of the research on selection comes from North America, with studies exploring the predictive validity of the MCAT, the GPA and, to a lesser extent, other aspects of the selection process. There have been fewer studies elsewhere, but increasingly researchers in Australia, the UK and the Netherlands are publishing in this area.

Research in student selection is replete with methodological difficulties and these are outlined in Box 2. The major issue is that there are few opportunities for truly randomised trials, as only successful applicants are available to be studied. As a result, research predominantly consists of correlation studies, examining how well selection scores correlate with desired outcomes in the medical course. Correlation studies are not able to infer causation, but are powerful if the correlation represents a significant proportion of common variance. Because selection processes choose only the high-scoring applicants, there is significant restriction of range, leading to weak correlations between student selection scores and in-program assessment. This range restriction can be adjusted for by using well established statistical techniques,20 but can never be completely overcome.

In addition to methodological issues, there are challenges in defining and categorising acceptable outcome measures, especially those that reflect non-academic performance such as professional behaviour.

With relatively limited sample sizes and significant variation in selection processes across universities and over time, large multicentre studies of selection processes have, so far, not been possible in Australia. However, there is some interest in using the large databases derived from UMAT and GAMSAT scores, and integrating these with data available from the Medical Student Outcome Database and Longitudinal Tracking Project (www.medicaldeans.org.au/projects-activities/msod).

Published research that takes methodological factors into consideration supports the use of selection processes that include non-academic criteria.9,21 For example, both national and international research shows that MMIs are reliable,4,19 and there is early evidence of their ability to predict future performance.4 A recent Australian study showed that providing skills-based training to interviewers significantly decreased variance in scoring of MMIs.22 Research evidence also shows that the correlation between scores in measures of non-academic performance and clinical performance increases during a medical course.23

One of the limitations of selection research has been restricting investigation to individual tools while neglecting the overall process of selection. It has at least been shown that a structured selection process provides better results than random selection by lottery, a method that has been used in the Netherlands.24 However, there has been no research into the optimal mix of instruments and how scores are best combined. It is unlikely there will be a single answer, as the context in which selection occurs significantly influences the type(s) of students selected for a given program.

Selection into medicine is a high-stakes activity and individual applicants are highly motivated to maximise their chances of selection. The possibility of candidates receiving coaching is an issue for all selection committees, although there is mixed evidence about its effectiveness. However, one study found no impact of coaching on interview performance in the Australian context.25

“Faking good”, that is, candidates giving false answers to present themselves in a favourable light, is a particular problem in interviews and also for personality tests.26 Previous studies have shown that this can affect hiring decisions in business,27 but its impact on selection into medicine has not yet been investigated. It is unlikely that applicants with personality disorders, who will probably demonstrate unprofessional behaviour during the medical course or as practising clinicians, will be detected by existing selection processes. As major psychopathological conditions are best diagnosed by a psychiatric interview, it is not feasible, practically or financially, to use this method in selection processes; rather, this is best dealt with by sustained observation of students across the course. As a consequence, medical schools are developing rigorous systems for remediating or excluding students whose behaviour is persistently unprofessional.28

Recent research has focused on broadening the repertoire of tools and processes used for selection. Personality testing, for example, is being explored as a tool for medical school selection, based on the assumption that it is easier and more reliable to use a written psychometric test than conduct an interview.29 The critical issue for clarification is the relationship between an applicant’s personality traits and their subsequent performance, as there is a requirement for a variety of personality types to meet the demands of the different careers within the profession. So far, little correlation has been found between interview scores and the “Big Five” personality traits — openness, conscientiousness, agreeableness, extroversion and neuroticism.30 Specific instruments, such as the Personal Qualities Assessment (www.pqa.net.au),5 have been trialled and will be used in selection for entry into an Australian medical school this year.

The concept of using centralised assessment centres for selection has been developed by organisational psychologists and personnel experts for a range of managerial and non-managerial occupations.31 An assessment centre allows an applicant to participate in various simulated activities and be observed by several trained assessors. The advantages are enhanced standardisation and greater efficiency for applicants. Selection centres have been extensively used for recruitment for postgraduate positions in the UK,32 and more recently in Australia,33 but not as yet for Australian medical school selection.

Another advance in selection is the outreach work done by medical schools to encourage disadvantaged secondary school students who would not normally consider applying for entry to medical courses. Programs in the UK, United States and Australia have been developed to provide students with experience of medical school and support with studies both before and in the early years of the medical course.

The range of methods now used to select students for medical schools in Australia has opened up opportunities for a career in medicine to applicants who previously would not have been selected or even considered applying. This has been accompanied by a greater appreciation of the importance of attributes other than academic ability, such as values and personal characteristics. However, to ensure that only the best applicants are selected, there is a pressing need for more evidence derived from long-term programmatic research.