How best to select medical students (ie, future doctors) is a topic that has rarely been out of the medical journals and the daily newspapers during the past 30 years. In this Journal in 1974, Campbell et al wrote: “Although mounting criticism and concern are expressed for the manner in which our medical students are selected, the status quo continues”.1 The status quo alluded to was selection of medical students solely on the basis of academic marks obtained in high school matriculation examinations. Fifteen years later, John Best, in one of his regular articles for the Journal,2 acknowledged that to be a good doctor required skills and personal qualities additional to academic aptitude, and wrote:

The University of Newcastle led a major change in medical education in Australia in the late 1970s by introducing problem-based learning, early clinical skills acquisition, community orientation, and the addition of personal qualities evaluation to the student selection process. The first Newcastle students were admitted in 1978. The Foundation Dean, David Maddison, anticipated the current debate and planned accordingly. The “Newcastle Experiment”3 involved half the students being admitted on the basis of academic marks alone (top 1%–2%), irrespective of personal qualities assessed by test and interview; the other half were admitted on the basis of personal qualities after applying an academic threshold (top 10%).

After a 9-year period, prior academic performance and personal qualities at the time of selection were compared with two outcome measures: a negative measure — non-completion of the course — and a positive one — graduation with honours. The results showed no relationship between either outcome and prior academic performance. However, there were important associations for both the negative and positive outcomes with personal qualities at the time of selection. The interview had the most predictive power in the selection procedure — those who did well in the areas of communication skills, motivation to be a doctor, and capacity to provide support to those in distress had a greater likelihood of completing their studies at medical school and of graduating with honours.4

The Newcastle Experiment was followed by another study, on interns in New South Wales hospitals, which demonstrated that Newcastle graduates at the beginning of internship were the equals of graduates of the other NSW medical schools in clinical competence, and had higher scores in the four personal characteristics evaluated5 — a desirable endpoint in the context of being a professional doctor.

It is probable that the findings documented in these two studies and others from Newcastle influenced selection procedures across the country, for by the mid to late 1990s nearly all Australian medical schools had begun to select students on the basis of both academic ability and personal qualities assessed by tests and interview.

Ten years down the track, the evaluation of personal qualities by interview is now being abandoned by some schools. Why is this happening? A study from the University of Queensland, reported in this issue of the Journal, gives some indication (→ Medical school selection criteria and the prediction of academic performance).6 Its major finding is that criteria other than prior academic grades only modestly predict academic performance in the medical course. That prior academic achievement should predict subsequent academic achievement is no surprise;7 indeed it is almost axiomatic. However, given the current view that graduating doctors should be more than just academically competent, it is of some concern that academic performance was the sole outcome measured in the study. In the present-day context of what we expect in a graduating doctor, the important finding in the Queensland study is the significant association of the interview scores with assessment outcome, especially the increased association for Year 4 (when, presumably, more clinically relevant matters are tested) compared with Year 1. The size of the effect is small, but medicine is accustomed to important small effects, such as the link between passive smoking and lung cancer. The finding that the written Graduate Australian Medical School Admissions Test (GAMSAT) does not predict within-course academic performance is very interesting, particularly given GAMSAT’s emphasis on scientific knowledge and understanding.

It is puzzling that the University of Queensland has decided to abandon the interview, given the significant results of their study. Even more puzzling is the stated intention to retain GAMSAT, which had no predictive power at all. In another recent study published in the Journal, GAMSAT was found to be a negative predictor of clinical reasoning.8

The most logical explanation for the proposed selection strategy is one founded on cost. No one questions the fact that having a selection process based on anything other than prior academic performance at school or university is expensive. It is expensive in time and effort for university staff, for community interviewers and for the applicants themselves. It is also politically expensive, especially when applicants with exceptionally high prior academic scores are not selected. The outrage expressed in a number of recent media reports9,10 reflects the passion with which sections of the public regard admission to medical school. Is the interview being abandoned because it is too expensive in terms of time and resources? Is GAMSAT being retained because it costs the medical school nothing, since applicants pay for it, and its numerical scores are a convenient way to differentiate between applicants?

The evidence from these two Australian medical schools,3-6 together with community expectations of the skills and competencies of a graduating doctor, supports the retention of methods to evaluate personal qualities during selection. However, this approach, especially the interview, comes at a cost. The question for medical schools is whether the effectiveness is worth the cost. If it is not, we should be honest with ourselves and the public and say that the cost is too much, even though it works.

My view is that we should continue to select medical students based on criteria that include desirable personal qualities, using procedures that have demonstrated reliability and validity. We also need to build into medical curricula objective barrier assessments of professional skills and personal qualities relevant to future medical practice. If we do this, we will not only graduate doctors who have the required skills, but we will have the appropriate outcome measures against which to evaluate our personal qualities selection procedure. If we elect not to take these two steps, we will be destined to go through yet another cycle of questioning selection methods. Campbell and colleagues’ 1974 comments1 will be reprised by others.