Teaching on the run tips 8: assessment and appraisal

Doctors find the terms “assessment”, “appraisal” and “evaluation” difficult and confusing. This is understandable given that there are no agreed definitions and the terms are commonly used interchangeably. As supervisors, we are asked to make judgements on the suitability of junior staff for medical registration, entry into vocational training programs or receiving the fellowship of a clinical college. In short, we are asked to assess junior staff. Clinicians also have roles as teachers and supervisors/mentors.1 As junior doctors both learn and are assessed on the job, they require feedback about their progress in learning outcomes and about their strengths and weaknesses, so that they can plan how to improve. All these processes are integrally linked, as shown by the learning cycle (“Tips 1”).1 Poor understanding of their role in assessing junior doctors’ performance may cause problems for some doctors. Furthermore, many of us have never been appraised or received feedback on how we are doing, yet we are expected to do it for our junior staff.

The following are some working definitions to help clinicians understand the roles and processes:2-6

Assessment: a judgement about how someone’s performance meets defined criteria. The standards are usually set by external bodies (eg, medical boards or colleges), and the result of the assessment will affect progress of trainees in their career. It is also known as “summative” assessment.

This means your recommendation to the hospital, medical board or college allows it to decide, “Yes, the trainee can progress to the next stage” or “No, the trainee has not satisfied the criteria”.

“Formative” assessment mimics the summative, but the purpose is to “inform” the learner of his or her progress before the summative assessment. Along with the quality of your observations, essential ingredients for accurate assessment are the explicit learning outcomes, which must be clearly stated. The assessment forms used should list the learning outcomes and provide a scale to help assessment.

Appraisal: a process that is primarily educational and developmental, in that it reviews current performance and develops plans to address the learning needs of an individual. It is jointly developed by the trainer and trainee and should be seen as confidential and non-threatening.

This involves you and the trainee having a discussion about your respective impressions of how the training is going, giving your advice, and jointly developing a plan on how to address any problems.

Although similar to a formative assessment, appraisal is usually much broader, including not only criteria that might be listed on the assessment form, but other things such as personal progress, career interests, and how the trainee is coping with the workload, study and family life. Appraisal is a key role of a good supervisor. Giving feedback on how junior doctors are doing and helping them to address concerns increase the likelihood that they will pass their assessment and feel fulfilled in professional life.

Evaluation: the trainee’s judgement of the trainer (clinician) or program (hospital, unit).

This gives the trainee the opportunity to tell you how good the training program was, enabling you to change and improve your practice.

If you really want to know how good you have been as a supervisor, evaluation should be collected, preferably anonymously, at a time that is separate from when you are giving feedback. A clinician who asks, “Why don’t you tell me how you’ve found the training program and then I’ll tell you how you’ve performed” won’t elicit much meaningful information.

As clinicians, we appraise and assess the same trainee. The information on which both are based, gathered while we work with the trainee, is the same. It might not be in the candidate’s best interests to reveal in an assessment confidential information gathered during an appraisal. On the other hand, a trainee needs to be honest, open and capable of self-assessment in order to benefit from appraisal. Most of the time there is little conflict between the processes of appraisal and assessment, as our trainees are keen to improve, but occasionally a conflict may arise. For instance, the clinician may be torn between the wish to support and the need to fail a poorly performing trainee, or may be unable to develop a good supervising relationship, which in turn may prejudice the assessment. In such circumstances, we should recognise the conflict and call for external help from, for example, a director of clinical training.

Key features of good in-training assessment2-4 and appraisal2,4 are:

• Clear outcomes and criteria

• Appropriate timing

• Accurate evidence

• Learner input

• Constructive, regular feedback.