Volume 178 - Issue 4
How good is the newly graduated doctor and can we measure it?
Author: H Thomas Aretz
Med J Aust 2003; 178 (4): 147-148. || doi: 10.5694/j.1326-5377.2003.tb05126.x
Published online: 17 February 2003
Published online: 17 February 2003
As medical curricula and the competencies required of new graduates evolve, evidence-based evaluation of these curricula should become routine
Over the last two to three decades, many universities and medical schools have initiated curriculum reforms. The factors cited as drivers of this worldwide reform include the ever-increasing knowledge base, shift in the burden of disease, advances in technology, changing expectations of patients and societies, easy access to healthcare information via the Internet, need for cost-effectiveness, the quality movement and awareness of medical errors, and changes in the learning environment.
As desired competencies of physicians have gone beyond factual knowledge and clinical skills, training programs need evaluation methods that measure these competencies
As societies struggle with their healthcare systems and re-examine the question "What is a good doctor and how do we measure it?", medical schools, teaching hospitals, professional societies and regulatory bodies try to provide guidelines, answers and leadership. As a consequence, there has been a proliferation of competency lists, and the movement towards "outcome-based education".1 One of the more meaningful lists of competencies was developed as part of the Education of Future Physicians of Ontario (EFPO) project2 (Box) — the first six of the eight competencies in this list resulted from patient input.
Competencies required of medical graduates*2
Medical expert–clinical decision maker
Communicator–educator–humanist–healer
Collaborator
Gatekeeper–resource manager
Learner
Health advocate
Scientist–scholar
The physician as a person
* Developed for the Education of Future Physicians of Ontario project.
Medical educators have therefore been asked to redefine the required knowledge base, set of skills, attitudes and experiences of physicians at all levels of training, while devising outcome metrics (assessments) that would allow meaningful feedback to individuals, schools, training programs and the system as a whole.3,4 The design and identification of these outcome metrics continue to be a challenge and reflect to a certain extent the diverse missions of medical schools:
As university students, medical students receive an advanced scientific degree, implying a depth of knowledge and mastery of scientific skills. Assessments and outcome metrics have traditionally concentrated on the scientific aspects of medicine, which appear narrow in light of the above-listed competencies.
Medical schools prepare their students to function as doctors for the rest of their lives. In most parts of the world, the government or a professional body certifies and licenses doctors, endorsing them as competent practitioners of medicine. Reliable metrics of the quality of medical practice remain a major challenge. Non-adherence to accepted medical guidelines has many reasons.5,6 However, awareness of and use of guidelines has been a measurable effect of one recent curricular change, involving problem-based and self-directed learning along with regular assessments of skills and attitudes.7
Medical schools also prepare their students to function as interns and junior medical officers after graduation. In essence, the directors of house-officer training programs and their colleagues are some of the earliest sentinels of medical student performance.
In this issue of the Journal (page 163), Dean and colleagues use the last aspect to collect performance data about recent graduates of the University of Sydney graduate-entry medical program.8 Using a self-assessment tool and collecting data from supervisors, they found that graduates of the new curriculum had better communication and teamwork skills, and that they approached patient care in a more holistic manner, while being more confident and willing to continue to learn. The graduates of the new program assessed their knowledge base as weaker than that of their traditionally trained colleagues, a fact that was supported by some, but not all, of their supervisors.
Despite the weaknesses of the study design acknowledged by the authors, these results reaffirm the value of follow-up studies as outcome metrics of medical education.9 As the "officially" desired competencies of physicians have gone beyond factual knowledge and clinical skills, training programs need to institute evaluation methods that measure these competencies.10 While more quantitative measures are developed, and increasing numbers of new curricula are introduced, better communication and feedback is needed between the various levels of medical education to guide this process.
Medical education is a continuum and a lifelong endeavour, and all its components — secondary schools, colleges, medical schools, training programs, practices, hospitals and professional societies — need to work together.5 As in any complex, interdependent system, frequent feedback is absolutely necessary; we need intermediate and surrogate markers to assess progress and allow us to take corrective steps. The present study provides one such marker, albeit limited.8 Similar data collection should be encouraged as a matter of routine to inform the system and improve the training of physicians.
References
- Harden RM. Developments in outcome-based education [editorial]. Med Teacher 2002; 24: 117-120.
- Neufeld VR, Maudsley RF, Pickering RJ, et al. Educating future physicians for Ontario. Acad Med 1998; 73: 1133-1148. i1082312
- Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002; 287: 226-235. i1082314
- Leach DC. Competence as a habit [editorial]. JAMA 2002; 287: 243-244. i1082316
- Newcomer LN. Physician, measure thyself. The price of physicians' autonomy is accountability. But how can anyone be accountable if performance is not measured? Health Affairs 1998; 17(4): 32-35. i1082318
- Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow practice guidelines? A framework for improvement. JAMA 1999; 282: 1458-1465. i1082320
- Shin JH, Haynes RB, Johnston ME. Effect of problem-based, self-directed undergraduate education on lifelong learning. CMAJ 1993; 6: 969-976. i1082322
- Dean SJ, Barrat A, Hendry GD, Lyon PMA. Preparedness for hospital practice among medical graduates from a problem-based, graduate-entry medical program. Med J Aust 2003; 178: 163-166.<eMJA full text>
- Jones A, McArdle PJ, O'Neill PA. Perceptions of how well graduates are prepared for the role of pre-registration house officers: a comparison of outcomes from a traditional and an integrated PBL curriculum. Med Educ 2002; 36: 16-25. i1082326
- Accreditation Council of Graduate Medical Education (ACGME) and American Board of Medical Specialties (ABMS). Toolbox of assessment methods. Version 1.1. Available at: http://www.acgme.org/outcome/2000 (accessed Nov 2002).