Achieving equal standards in medical student education: is a national exit examination the answer?

To the Editor: The recent article by Koczwara and colleagues proposing a national exit examination for medical students1 prompted me to recall a 1970 trial of a national examination in surgery.2 Seven of the then eight medical schools participated. Interstate differences were wide for some questions; separate analyses of the 15 teaching hospitals showed variation to be even wider within a university than between universities. Do local differences still undermine the validity of a national examination?

It is still uncertain what is actually tested by questions on paper. Context-free, standardised questions and answers assume clinical teaching and practice are standardised. However, clinical teachers writing examination items know well that many colleagues choose the “wrong” answer. Consensus may be imposed on those who differ. Teachers then forget their disparity, but expect candidates to choose only one “true” answer!

Clinical performance is interactive, multifaceted, situation-specific and value-laden. Complex judgement and decision-making cannot be measured by ticking predetermined boxes. Clinical experts develop personal subsets of specific evidence, and seek different data for diagnosis and management. But separate, context-free tasks, as in an objective structured clinical examination (OSCE), naively assume they do not.3 OSCE even standardises scoring; examiners become recorders rather than assessors.

Reductionist standardisation reflects a pseudoscientific attempt to apply objectivity, consistency and precision to complex human interactions around incomplete evidence and uncertainty, approximations, judgements, trade-offs and locally-determined decisions.4 Internal consistency of measuring instruments does not confer external validity in real world clinical practice.

The inexorable growth of medical knowledge and technological opportunities continuously expands “what every doctor should know”. Medical learning today embraces a mix of science-based, problem-based and work-based learning experiences, with community-based experiences5 increasingly included. In their recent article, Koczwara and colleagues identified gaps in oncology education,2 a field ranging from molecular processes to euthanasia. Is oncology managed and taught consistently across different medical schools and hospitals across Australia? Which facets would you test in a national exit examination?6

Clinical performance today includes patient/person management, case management, health system management and self-management. Clinicians can judge student performance consistently. However, formal clinical examinations lack the range of cases and open-ended time that allow examiners to observe all the patient-care skills espoused by today’s curricula.7 Assessment of performance in case management and procedural skills within hospital practice can be conducted simpy by paired examiners.8