Benchmarking in Australia using the International Foundations of Medicine Clinical Science Examination

Demonstration of consistent standards in medical education is an important response to the increasing mobility of the workforce and the propagation of new medical schools around the world.1,2 Five Australian medical schools (at the universities of Adelaide, Melbourne, Sydney, Queensland and Western Australia) collaborated to perform a comparison across the subdisciplines included on the 2014 International Foundations of Medicine Clinical Science Examination (IFOM CSE).3

In 2014, a total of 1425 (of 1448) students in the final year of their primary medical degree at the participating Australian institutions took the examination. All schools used this examination as a formative assessment, although participation was compulsory at three schools. The lowest participation rate was 93% (155/167). The tests were delivered at different times of the academic year, with various administration conditions, including mode of delivery (electronic and paper-based).

The IFOM CSE consists of 160 multiple-choice questions created from items drawn from the United States Medical Licensing Examination (USMLE), which is produced by the US National Board of Medical Examiners (NBME). Results were collated into 18 profile scores, and the overall mean score at the five schools ranged from 502 to 549 (SD, 54–76). Eight students were excluded from analysis on the basis of a non-serious attempt, with a scale score ≤ 308 reflecting less than 30% of answers correct and guessing behaviour.

The NBME provides two comparisons to assist score interpretation — the mean and standard deviation for the International Comparison Group (ICG), which is made up of all participants in the test worldwide, and the equivalent passing score for Step 2 Clinical Knowledge (CK) of the USMLE, an extremely high-stakes examination in the US.4 While all Australian schools had a higher mean IFOM CSE score than the ICG, only 22% of students (311/1425) met the standard of competence equivalent to Step 2 CK of the USMLE. This result may be attributable to the difference in stakes between the two examinations, as well as differences in health systems and disease prevalence between the US and Australia.

As the collaboration shared de-identified data to protect the anonymity of schools, individual school rates for reaching the USMLE standard could not be assessed. There were too many differences in methods and timing of test administration to make substantive claims with aggregated de-identified data. The students from one school performed significantly lower (Z = − 3.34, P < 0.01) and those from another performed significantly higher (Z = 2.49, P < 0.01) than the overall mean score, despite fewer students in the higher performing school reaching the Step 2 CK standard. Both these schools had non-compulsory participation in the IFOM CSE.

To ensure that comparisons by subdiscipline across the medical schools reflected curriculum effects rather than student abilities, the scores were anchored to the overall mean score for the five schools (ie, 528). The adjusted subscores by subdiscipline showed very little variation between the five schools.

Longitudinal data obtained over time will reveal whether our results reflect stable curriculum effects. The Australian medical schools that have participated in the IFOM CSE have found it useful and informative.5 By sharing our results to conduct these comparisons, we have identified further how these data may inform curriculum decisions. For example, if the school that performed better than expected on the preventive medicine and health maintenance section is shown to consistently do so, course content could be shared to allow improvements in medical programs for all involved in the collaboration.