To the Editor: We are concerned about aspects of the study of selection predictors of medical school performance in Australian medical schools by Groves et al,1 and how they may be interpreted.

A “voluntary” response rate of 13.6% is very small and unlikely to be representative, and selection or response bias is likely. Although noted by the authors, this fundamental flaw may be overlooked in interpreting the results. This aside, of what value is the outcome measure of clinical reasoning skills among students, some of whom are only in second year, and whom we would not expect to have developed substantial expertise in clinical reasoning at this early stage?

More significantly, we consider that the research question itself may be flawed, as we are not at all convinced that selection scores are intended to predict relative performance in programs. Groves et al, of course, show that, in fact, they do not.

Selection serves two purposes, one obvious and the other questionable. The first is to reduce the large pool of well qualified students to the available number of places. The second is the Holy Grail of selection: attempting to select those best suited to success in medicine. The key problem is that nobody can fully define, let alone measure, such success, other than in the negative terms of professional misconduct after graduation.2 Moreover, medicine offers a wide range of careers requiring different attributes, making a broad range of entry characteristics beneficial to the profession and the community at large, and putting in question the pursuit of a profile of critical criteria.

Further research is needed, involving large, representative samples to confirm or discount putative outcome measures, but most medical students succeed in medical school and do not create problems as doctors, indicating that we are currently getting more of this right than wrong. If the predictive value of entry attributes for (often questionable) outcomes is modest at best, and if we teach and assess communication skills, clinical reasoning and professionalism during the programs, does it not make sense to focus our energies and resources more effectively there, and relinquish our continuing anxiety over selection? We should remember what motivated the flurry of activity towards more complex selection processes in the first place: concern over communication by doctors, failure in aspects of professionalism, and to a much lesser extent, clinical competence.

In reply: The limitations of our study have been acknowledged, and we agree with Parker and colleagues that further research is needed. However, their proposition that selection scores may not be intended to predict relative performance in programs seems at odds with their recommendation to select principally on the basis of academic performance. If selection scores do not provide predictability, why bother establishing any criteria at all? Why not simply use a lottery system, as they have been claimed to be equally effective?1

Although our study did not set out to evaluate the relative merits of cognitive versus non-cognitive criteria for selecting medical students, surely the first question to be decided is whether non-cognitive characteristics, including interpersonal skills, attitudes and behaviour, are important in medical practice and should be considered in selecting future doctors. If so, then the next question is how best to select students with these characteristics or, at least, the capacity to develop them during their medical training. The selection processes of most medical schools indicate that the answer to the first question is “yes”. In answering the second question, not only the validity of the chosen method, but its feasibility in terms of cost and effectiveness need to be considered.