Objective: To assess the accuracy and variability of clinicians’ estimates of pre-test probability for three common clinical scenarios.
Design: Postal questionnaire survey conducted between April and October 2001 eliciting pre-test probability estimates from scenarios for risk of ischaemic heart disease (IHD), deep vein thrombosis (DVT), and stroke.
Participants and setting: Physicians and general practitioners randomly drawn from College membership lists for New South Wales and north-west England.
Main outcome measures: Agreement with the “correct” estimate (being within 10, 20, 30, or > 30 percentage points of the “correct” estimate derived from validated clinical-decision rules); variability in estimates (median and interquartile ranges of estimates); and association of demographic, practice, or educational factors with accuracy (using linear regression analysis).
Results: 819 doctors participated: 310 GPs and 288 physicians in Australia, and 106 GPs and 115 physicians in the UK. Accuracy varied from about 55% of respondents being within 20% of the “correct” risk estimate for the IHD and stroke scenarios to 6.7% for the DVT scenario. Although median estimates varied between the UK and Australian participants, both were similar in accuracy and showed a similarly wide spread of estimates. No demographic, practice, or educational variables substantially predicted accuracy.
Conclusions: Experienced clinicians, in response to the same clinical scenarios, gave a wide range of estimates for pre-test probability. The development and dissemination of clinical decision rules is needed to support decision making by practising clinicians.
- 1. None identified.
- 2. Jaeschke R, Guyatt G, Lijmer J. Diagnostic tests. In: Guyatt G, Rennie D, editors. Users’ guides to the medical literature. A manual for evidence-based practice. Chicago: AMA Press, 2002.
- 3. Letelier LM, Weaver B, Montori V. Diagnosis. Examples of likelihood ratios. In: Guyatt G, Rennie D, editors. Users’ guides to the medical literature. A manual for evidence-based practice. Chicago: AMA Press, 2002.
- 4. Richardson WS. Five uneasy pieces about pre-test probability. J Gen Intern Med 2002; 17: 891-892.
- 5. Sackett DL, Haynes RB, Guyatt GH, Tugwell P, editors. Clinical epidemiology. A basic science for clinical medicine. 2nd edition. Boston: Little Brown, 1991: 100.
- 6. Bobbio M, Fubini A, Detrano R, et al. Diagnostic accuracy of predicting coronary artery disease related to patients’ characteristics. J Clin Epidemiol 1994; 47: 389-395.
- 7. Bobbio M, Detrano R, Shandling AH, et al. Clinical assessment of the probability of coronary artery disease: judgmental bias from personal knowledge. Med Decis Making 1992; 12: 197-203.
- 8. Lyman GH, Balducci L. The effect of changing disease risk on clinical reasoning. J Gen Intern Med 1994; 9: 488-495.
- 9. Dolan JG, Bordley DR, Mushlin AI. An evaluation of clinicians subjective prior probability estimates. Med Decis Making 1986; 6: 216-223.
- 10. Diamond GA, Forrester JS. Analysis of probability as an aid in the clinical diagnosis of coronary artery disease. N Engl J Med 1979; 300: 1350-1358.
- 11. Wells PA, Anderson DR, Bormanis J, et al. Value of assessment of pretest probability of deep-vein thrombosis in clinical management. Lancet 1997; 350:1795-1798.
- 12. Wolf PA, D’Agostino RB, Belanger AJ, Kannel WB. Probability of stroke: a risk profile from the Framingham Study. Stroke 1991; 22: 312-318.
- 13. JMP [statistical software], version 4, Cary, NC: SAS Institute Inc, 2001.
- 14. Lipinski M, Do D, Froelicher V, et al. Comparison of exercise test scores and physician estimation in determining disease probability. Arch Intern Med 2001; 161: 2239-2244.
- 15. McManus RJ, Mant J, Meulendijks CFM, et al. Comparison of estimates and calculations of risk of coronary heart disease by doctors and nurses using different calculation tools in general practice: cross sectional study. BMJ 2002; 324: 459-464.
- 16. Noguchi Y, Matsui K, Imura H, et al. Quantitative evaluation of the diagnostic thinking process in medical students. J Gen Intern Med 2002; 17: 848-853.
- 17. Steurer J, Fischer JE, Bachmann LM, et al. Communicating accuracy of tests to general practitioners: a controlled study. BMJ 2002; 324: 824-826.
- 18. Stevens J. Applied multivariate statistics for the social sciences. 3rd edition. Mahwah, NJ: Lawrence Erlbaum Publishers, 1998: 72.
- 19. Shim JH, Haynes RB, Johnson ME. Effect of problem-based, self-directed, undergraduate education on life-long learning. Can Med Assoc J 1993; 148: 969-976.
- 20. Norman GR, Shannon SI. Effectiveness of instruction in critical appraisal skills: a critical appraisal. Can Med Assoc J 1998; 158: 177-181.
- 21. McGettigan P, Sly K, O'Connell D, et al. The effects of information framing on the practices of physicians. J Gen Intern Med 1999; 14: 633-642.
- 22. Brassey J, Elsyn G, Price C, Kinnersley P. Just in time information for clinicians: a questionnaire evaluation of the ATTRACT Project. BMJ 2001; 322: 529-530.
- 23. Stiell IG, McKnight RD, Greenberg GH, et al. Implementation of the Ottawa ankle rules. JAMA 1994; 271: 827-832.
- 24. Stiell IG, Wells GA, Vandemheem KL, et al. The Canadian C-spine rule for radiography in alert and stable trauma patients. JAMA 2001; 286: 1841-1848.
Publication of your online response is subject to the Medical Journal of Australia's editorial discretion. You will be notified by email within five working days should your response be accepted.