Connect
MJA
MJA

Generating pre-test probabilities: a neglected area in clinical decision making

John R Attia, David W Sibbritt, Ben D Ewald, Balakrishnan R Nair, Neil S Paget, Rod F Wellard, Lesley Patterson and Richard F Heller
Med J Aust 2004; 180 (9): 449-454.

Summary

Objective: To assess the accuracy and variability of clinicians’ estimates of pre-test probability for three common clinical scenarios.

Design: Postal questionnaire survey conducted between April and October 2001 eliciting pre-test probability estimates from scenarios for risk of ischaemic heart disease (IHD), deep vein thrombosis (DVT), and stroke.

Participants and setting: Physicians and general practitioners randomly drawn from College membership lists for New South Wales and north-west England.

Main outcome measures: Agreement with the “correct” estimate (being within 10, 20, 30, or > 30 percentage points of the “correct” estimate derived from validated clinical-decision rules); variability in estimates (median and interquartile ranges of estimates); and association of demographic, practice, or educational factors with accuracy (using linear regression analysis).

Results: 819 doctors participated: 310 GPs and 288 physicians in Australia, and 106 GPs and 115 physicians in the UK. Accuracy varied from about 55% of respondents being within 20% of the “correct” risk estimate for the IHD and stroke scenarios to 6.7% for the DVT scenario. Although median estimates varied between the UK and Australian participants, both were similar in accuracy and showed a similarly wide spread of estimates. No demographic, practice, or educational variables substantially predicted accuracy.

Conclusions: Experienced clinicians, in response to the same clinical scenarios, gave a wide range of estimates for pre-test probability. The development and dissemination of clinical decision rules is needed to support decision making by practising clinicians.

Please login with your free MJA account to view this article in full

  • John R Attia1
  • David W Sibbritt2
  • Ben D Ewald3
  • Balakrishnan R Nair4
  • Neil S Paget5
  • Rod F Wellard6
  • Lesley Patterson7
  • Richard F Heller8

  • 1 Centre for Clinical Epidemiology and Biostatistics, University of Newcastle, Newcastle, NSW.
  • 2 Department of Medicine, John Hunter Hospital, New Lambton, NSW.
  • 3 The Royal Australasian College of Physicians, Sydney, NSW.
  • 4 Royal Australian College of General Practitioners, South Melbourne, VIC.
  • 5 Evidence for Population Health Unit, University of Manchester, Manchester, UK.

Correspondence: 

Competing interests:

None identified.

  • 1. None identified.
  • 2. Jaeschke R, Guyatt G, Lijmer J. Diagnostic tests. In: Guyatt G, Rennie D, editors. Users’ guides to the medical literature. A manual for evidence-based practice. Chicago: AMA Press, 2002.
  • 3. Letelier LM, Weaver B, Montori V. Diagnosis. Examples of likelihood ratios. In: Guyatt G, Rennie D, editors. Users’ guides to the medical literature. A manual for evidence-based practice. Chicago: AMA Press, 2002.
  • 4. Richardson WS. Five uneasy pieces about pre-test probability. J Gen Intern Med 2002; 17: 891-892.
  • 5. Sackett DL, Haynes RB, Guyatt GH, Tugwell P, editors. Clinical epidemiology. A basic science for clinical medicine. 2nd edition. Boston: Little Brown, 1991: 100.
  • 6. Bobbio M, Fubini A, Detrano R, et al. Diagnostic accuracy of predicting coronary artery disease related to patients’ characteristics. J Clin Epidemiol 1994; 47: 389-395.
  • 7. Bobbio M, Detrano R, Shandling AH, et al. Clinical assessment of the probability of coronary artery disease: judgmental bias from personal knowledge. Med Decis Making 1992; 12: 197-203.
  • 8. Lyman GH, Balducci L. The effect of changing disease risk on clinical reasoning. J Gen Intern Med 1994; 9: 488-495.
  • 9. Dolan JG, Bordley DR, Mushlin AI. An evaluation of clinicians subjective prior probability estimates. Med Decis Making 1986; 6: 216-223.
  • 10. Diamond GA, Forrester JS. Analysis of probability as an aid in the clinical diagnosis of coronary artery disease. N Engl J Med 1979; 300: 1350-1358.
  • 11. Wells PA, Anderson DR, Bormanis J, et al. Value of assessment of pretest probability of deep-vein thrombosis in clinical management. Lancet 1997; 350:1795-1798.
  • 12. Wolf PA, D’Agostino RB, Belanger AJ, Kannel WB. Probability of stroke: a risk profile from the Framingham Study. Stroke 1991; 22: 312-318.
  • 13. JMP [statistical software], version 4, Cary, NC: SAS Institute Inc, 2001.
  • 14. Lipinski M, Do D, Froelicher V, et al. Comparison of exercise test scores and physician estimation in determining disease probability. Arch Intern Med 2001; 161: 2239-2244.
  • 15. McManus RJ, Mant J, Meulendijks CFM, et al. Comparison of estimates and calculations of risk of coronary heart disease by doctors and nurses using different calculation tools in general practice: cross sectional study. BMJ 2002; 324: 459-464.
  • 16. Noguchi Y, Matsui K, Imura H, et al. Quantitative evaluation of the diagnostic thinking process in medical students. J Gen Intern Med 2002; 17: 848-853.
  • 17. Steurer J, Fischer JE, Bachmann LM, et al. Communicating accuracy of tests to general practitioners: a controlled study. BMJ 2002; 324: 824-826.
  • 18. Stevens J. Applied multivariate statistics for the social sciences. 3rd edition. Mahwah, NJ: Lawrence Erlbaum Publishers, 1998: 72.
  • 19. Shim JH, Haynes RB, Johnson ME. Effect of problem-based, self-directed, undergraduate education on life-long learning. Can Med Assoc J 1993; 148: 969-976.
  • 20. Norman GR, Shannon SI. Effectiveness of instruction in critical appraisal skills: a critical appraisal. Can Med Assoc J 1998; 158: 177-181.
  • 21. McGettigan P, Sly K, O'Connell D, et al. The effects of information framing on the practices of physicians. J Gen Intern Med 1999; 14: 633-642.
  • 22. Brassey J, Elsyn G, Price C, Kinnersley P. Just in time information for clinicians: a questionnaire evaluation of the ATTRACT Project. BMJ 2001; 322: 529-530.
  • 23. Stiell IG, McKnight RD, Greenberg GH, et al. Implementation of the Ottawa ankle rules. JAMA 1994; 271: 827-832.
  • 24. Stiell IG, Wells GA, Vandemheem KL, et al. The Canadian C-spine rule for radiography in alert and stable trauma patients. JAMA 2001; 286: 1841-1848.

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Responses are now closed for this article.