Evidence-Based Medicine

Jonathan C Craig, Les M Irwig and Martin R Stockler

MJA 2001; 174: 248-253

Abstract - The tools of EBM - Does it improve outcomes? - The future of EBM - Conclusions - Acknowledgements - References - Authors' details
Register to be notified of new articles by e-mail - Current contents list - More articles on Statistics, epidemiology and research design

Abstract

• Evidence-based medicine (EBM) integrates clinical experience and patient values with the best available research information.
  
• There are four steps in incorporating the best available research evidence in decision making: asking answerable questions; accessing the best information; appraising the information for validity and relevance; and applying the information to patient care.
  
• Applying EBM to individual patients requires drawing up a balance sheet of benefits and harms based on research and individual patient data.
  
• The most realistic and efficient use of EBM by clinicians at the point of care involves accessing and applying valid and relevant summaries of research evidence (evidence-based guidelines and systematic reviews).
  
• The future holds promise for improved primary research, better EBM summaries, greater access to these summaries, and better implementation systems for evidence-based practice.
  
• Computer-assisted decision support tools for clinicians facilitate integration of individual patient data with the best available research data.

Imagine yourself in the following situations:

• As a general practitioner you see a 72-year-old asymptomatic man who wants to know whether he should be screened for colorectal cancer. What do you say?

• As a nephrologist you see a 50-year-old man with progressive renal failure due to glomerulonephritis. Should you advise treatment with cyclosporin A?

• As an interventional radiologist or general surgeon should you be using antibiotic- or antiseptic-impregnated central venous lines to prevent line-associated sepsis?

Do you know enough of the research evidence to provide sensible answers to these questions? Even if you know today's answers, chances are that they will change as much in the next five years as they have changed in the past five years.^1-4 Therein lies the challenge — keeping up with new research information and incorporating it into clinical decision making. This is the task of evidence-based medicine (EBM).

In a survey of 625 office-based primary-care physicians and 100 physician opinion leaders in the United States, nearly two-thirds reported that the current volume of scientific information was unmanageable.⁴ When the researchers asked about the physicians' knowledge of important recent medical advances, they found deficiencies that would adversely affect patient care. Since 1989, when the above study was published, total biomedical knowledge has probably increased by about 50%.⁵ In addition to identifying a challenge, EBM also provides tools to find, appraise and apply research evidence better.⁶ These tools are relevant to all users of health information — clinicians, patients and policymakers — but our focus will be on helping clinicians make better use of EBM tools.

The tools of evidence-based medicine

The EBM process — incorporating the best available research evidence in decision making — has four steps:

• asking answerable questions;

• accessing the best information;

• appraising the information for validity and relevance; and

• applying the information to patient care.

Asking answerable questions

Accept that you may not know: While the knowledge explosion continues, making assumptions about the certainty of our knowledge base is risky. Studies of information needs show that one to two questions are generated for each outpatient consultation and five questions for each inpatient consultation.^8,9 About a third of these questions are about treatment of a specific condition, and a quarter are about diagnosis.¹⁰ EBM tools help answer these "foreground" questions,¹¹ which are specific and relevant to clinical decision making. Other questions concerning basic biological processes, or "background" questions (questions beginning with What is . . . ? and How does . . . ?), are better answered by standard textbooks. For these the EBM framework is not particularly helpful.

Framing the question: population, intervention, comparator, outcome: Once the clinical question has been identified, it then needs to be put into a searchable and answerable form. This consists of four parts:

• a population with a clinical problem;

• an intervention or exposure;

• the comparator intervention or exposure; and

• the outcomes.¹¹

Such questions are specific, and should focus on patient-centred or clinically important outcomes, rather than laboratory-based or surrogate outcomes that do not always correspond with patient benefit.¹²

For example, the question posed by the 72-year-old man wanting to know about colorectal cancer screening could be rephrased as: "In asymptomatic people at average risk of colorectal cancer (population), does screening by faecal occult blood testing (intervention) reduce mortality from colorectal cancer (outcome) compared with routine care without screening (comparator)?".

Aetiology, prognosis, diagnosis or intervention? This four-part question framework can be applied to all types of foreground questions asked by healthcare providers and consumers — Why me? (aetiology), What's wrong with me? (diagnosis), What will to happen to me? (prognosis), and How will intervention change outcome? (intervention). Examples of each type of question are given in Box 1. Because most questions asked by patients and clinicians are about interventions, we will focus on treatment.

Accessing the best available information

Summarised primary research — evidence-based guidelines and systematic reviews: The ideal information source is valid (contains high quality data), relevant (clinically applicable), comprehensive (has data on all benefits and harms of all possible interventions), and is user-friendly (is quick and easy to access and use).

The recent growth of EBM has provided more useful information sources (Box 2), and better access to these information sources (Boxes 2 and 3). Primary research data can now be organised into systematic reviews and evidence-based guidelines. For treatment questions, systematic reviews typically bring together, summarise and synthesise data from randomised controlled trials of a single intervention. Because many interventions are usually possible for the same clinical problem, systematic reviews of these interventions can be further summarised and combined in the form of an evidence-based guideline. To be most useful to clinicians, guidelines should also include diagnostic and prognostic research which provides some guidance for individualising therapy based on disease severity.¹³ Guidelines and systematic reviews can be stand-alone products, or, more usefully, can be organised into compendia.

Primary research — if quality summarised research is not available: When relevant systematic reviews or evidence-based guidelines are unavailable, or if they fail the quick critical appraisal test outlined in the following section, the clinician will need to find primary research studies. Although a randomised controlled trial is the best study type to assess the effects of a healthcare intervention, it is not the best study design to determine the accuracy of a diagnostic test or the prognosis of a condition, and is frequently not feasible for questions of aetiology.

Box 1 includes the ideal primary study design for each type of question, along with the appropriate methodological terms that help focus MEDLINE searches on these studies. For users unfamiliar with methodological filters, the free website for MEDLINE, PubMed, has a "clinical queries" option which allows users to select the content area and the type of question (therapy, diagnosis, aetiology or prognosis). The program then automatically incorporates the methodological filters into the search.¹⁴

Appraisal of quality and clinical relevance

Why quality assessment is needed: Having found the research information, the user then needs to critically appraise the study or studies. Publication does not guarantee quality, and poor-quality studies tend to overestimate the benefits of interventions by about 30%¹⁵ — enough to make ineffective interventions appear effective. Likewise, poor-quality studies of diagnostic tests overestimate the accuracy of the test they are evaluating.¹⁶

Tools for critical appraisal: Useful tools for critical appraisal developed for the National Health and Medical Research Council (NHMRC)^17,18 are summarised in Boxes 3 and 4. They ask three questions:

• How strong is the evidence?

• How big is the effect?

• Does the effect matter to patients?

The strength of evidence incorporates the appropriateness of the study design (often called level of evidence), the quality of the study's design and reporting (was bias minimised?) and the statistical precision of the results (could the results be explained by chance?). A few seconds scanning an abstract to see how well it rates on these criteria is often enough to indicate whether the study is worth reading.

If the initial scan suggests that the results may be reliable and important, then critical appraisal means focusing on the methods section to see how the study was done (not to see what statistical tests were used, such as whether a ² or t-test was used) and on the results, particularly the figures and tables.

Applying the research evidence to decision making

1. Make a balance sheet of the benefits and harms of the intervention

All outcomes (both beneficial and harmful) that are important to the patient and influenced by the intervention need to be considered.

For example, for the man with progressive kidney disease in our second scenario, we would need to consider the possible benefits of cyclosporin (reducing the need for dialysis) alongside the possible harms (gum hypertrophy, hypertension and hypertrichosis).

2. From research data, quantify the likelihood of benefits and harms in relative terms

How likely is it that the benefits and harms will affect an individual patient? To estimate this we need to know the average effect of the treatment from systematic reviews (or trials, if systematic reviews are not available) and whether the effect varies according to patient and disease factors or whether it is relatively constant and independent of these factors. This type of information comes from subgroup analyses of systematic reviews and large trials. The benefits and harms of interventions are generally best expressed in relative terms (such as relative risks), because the relative effect is often stable across many different patient subgroups.

In the trial of cyclosporin for progressive kidney disease, the relative risk of needing dialysis was 0.4 (the risk of dialysis was 0.4 times lower in those treated with cyclosporin than in those not treated with it), but the study was too small to determine whether the effect varied in different subgroups.

3. Convert the relative benefits and harms into absolute terms for your patient using the patient's specific characteristics

If the relative beneficial effect of treatment is stable across patients at different levels of risk from their disease, then those at greatest risk will have the most to gain from treatment, and those at least risk from their disease will have the least to gain. The absolute benefit of treatment (how much they have to gain) can be calculated by combining the relative effect of treatment (from randomised trials and systematic reviews) with the risk of the outcome without treatment (from cohort studies of prognosis). This is demonstrated in Box 5 using two groups of patients with kidney disease treated with cyclosporin.

While valid data from trials about the average benefit of a treatment are important, we also need valid data (preferably local) about the prognosis of patients without treatment to estimate the absolute benefit for any particular patient.

To return to our example, the patient's renal function, blood pressure and degree of proteinuria indicate that he belongs to the low risk group and has a probability of needing dialysis over the next few years of about 10%, which can be reduced to 4% with treatment.²⁰ The same logic can be used to calculate the risk of treatment-related harms.

4. Decide whether the benefits outweigh the harms

Having listed all benefits and harms of an intervention and assigned some likelihood for each outcome based on research and individual patient data, the next step is to determine whether, on balance, the treatment is likely to do more good than harm. If the various benefits and harms are roughly equivalent, then this is relatively easy. For example, in weighing up the benefits and harms of thrombolytic therapy for myocardial infarction, it is reasonable to count deaths prevented (from myocardial infarction) as equal to deaths caused (from cerebral haemorrhage) — they are equally undesirable. But not all outcomes are equal. How does a stroke prevented by aspirin compare with a gastrointestinal haemorrhage caused by it? The differential desirability of outcomes can be measured formally, preferably by patients, but more often this integration of probabilities and preferences is informal.

Does providing evidence-based care to patients improve outcomes?

The future of EBM

Better information systems at the point of care

Ultimately, given the complexity of the data, widespread use of high-quality evidence requires computer-based information management systems. Such computerised decision support systems for clinicians have already been developed, and are available to help clinicians provide better care.^26-28 The clinician's role is to use clinical judgement to integrate the best available research information and the patient's unique circumstances and preferences into a plan of management.

Better primary research

As well as unequal coverage of diseases, there is also unequal coverage of question types. While there are many randomised controlled trials of treatments, there are too few high quality studies of diagnostic tests, prognoses and interventions to help clinicians use research information more effectively.

The research data relevant to the questions in the Introduction are given in Box 7.

Conclusions

• To use evidence summaries in clinical practice;

• To help develop and update selected systematic reviews or evidence-based guidelines in their area of expertise; and

• To enrol patients in studies of treatment, diagnosis and prognosis on which medical practice is based.

Acknowledgements

References

1. Towler B, Irwig L, Glasziou P, et al. A systematic review of the effects of screening for colorectal cancer using the faecal occult blood test, hemoccult. BMJ 1998; 317: 559-565.
2. Cattran DC, Appel GB, Hebert LA, et al. A randomized trial of cyclosporine in patients with steroid-resistant focal segmental glomerulosclerosis. North America Nephrotic Syndrome Study Group. Kidney Int 1999; 56: 2220-2226.
3. Veenstra DL, Saint S, Saha S, et al. Efficacy of antiseptic-impregnated central venous catheters in preventing catheter-related bloodstream infection: a meta-analysis. JAMA 1999; 281: 261-267.
4. Williamson JW, German PS, Weiss R, et al. Health science information management and continuing education of physicians. A survey of U.S. primary care practitioners and their opinion leaders. Ann Intern Med 1989; 110: 151-160.
5. Wyatt J. Uses and sources of medical knowledge. Lancet 1991; 338: 1368-1372.
6. Sackett DL, Straus SE, Richardson WS, et al. Evidence-based medicine: how to practice and teach EBM. 2nd edition. New York: Churchill Livingston, 2000.
7. Scott I, Heyworth R, Fairweather P. The use of evidence-based medicine in the practice of consultant physicians. Results of a questionnaire survey. Aust N Z J Med 2000; 30: 319-326.
8. Covell DG, Uman GC, Manning PR. Information needs in office practice: are they being met? Ann Intern Med 1985; 103: 596-599.
9. Osheroff JA, Forsythe DE, Buchanan BG, et al. Physicians' information needs: analysis of questions posed during clinical teaching. Ann Intern Med 1991; 114: 576-581.
10. Smith R. What clinical information do doctors need? BMJ 1996; 313: 1062-1068.
11. Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club 1995; 123: A12-A13.
12. Bucher HC, Guyatt GH, Cook DJ, et al. Users' guides to the medical literature: XIX. Applying clinical trial results. A. How to use an article measuring the effect of an intervention on surrogate end points. Evidence-Based Medicine Working Group. JAMA 1999; 282: 771-778.
13. National Health and Medical Research Council. Clinical practice guidelines for the management of early breast cancer. 2nd edition. Canberra: NHMRC, 2000. <http://www.health.gov.au/nhmrc/advice/pdfcover/eabrscov.htm>
14. Hunt DL, Jaeschke R, McKibbon KA. Users' guides to the medical literature: XXI. Using electronic health information resources in evidence-based practice. JAMA 2000; 283: 1875-1879.
15. Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998; 352: 609-613.
16. Lijmer JG, Mol BW, Heisterkamp S, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 1999; 282: 1061-1066.
17. National Health and Medical Research Council. How to use the evidence: assessment and application of scientific evidence. Canberra: NHMRC, 2000.
18. Liddle J, Williamson M, Irwig L. Method for evaluating research and guideline evidence. Sydney: NSW Health, 1996.
19. Glasziou PP, Irwig LM. An evidence based approach to individualising treatment. BMJ 1995; 311: 1356-1359.
20. Chitalia VC, Wells JE, Robson RA, et al. Predicting renal survival in primary focal glomerulosclerosis from the time of presentation. Kidney Int 1999; 56: 2236-2242.
21. Mitchell JB, Ballard DJ, Whisnant JP, et al. What role do neurologists play in determining the costs and outcomes of stroke patients? Stroke 1996; 27: 1937-1943.
22. Soumerai SB, McLaughlin TJ, Spiegelman D, et al. Adverse outcomes of underuse of beta-blockers in elderly survivors of acute myocardial infarction. JAMA 1997; 277: 115-121.
23. Krumholz HM, Radford MJ, Ellerbeck EF, et al. Aspirin for secondary prevention after acute myocardial infarction in the elderly: prescribed use and outcomes. Ann Intern Med 1996; 124: 292-298.
24. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 1993; 342: 1317-1322.
25. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ 1995; 153: 1423-1431.
26. Walton R, Dovey S, Harvey E, Freemantle N. Computer support for determining drug dose: systematic review and meta-analysis. BMJ 1999; 318: 984-990.
27. Chatellier G, Colombet I, Degoulet P. Computer-adjusted dosage of anticoagulant therapy improves the quality of anticoagulation. Medinfo 1998; 9 Pt 2: 819-823.
28. Montgomery AA, Fahey T. A systematic review of the use of computers in the management of hypertension. J Epidemiol Community Health 1998; 52: 520-525.
29. Early Breast Cancer Triallists' Collaborative Group. Tamoxifen for early breast cancer: an overview of the randomised trials. Lancet 1998; 351: 1451-1467.
30. Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996; 276: 637-639.
31. Moher D, Cook DJ, Eastwood S, et al. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 1999; 354: 1896-1900.

Authors' details

Jonathan C Craig, MM(ClinEpi), PhD, FRACP, Senior Lecturer; Paediatric Nephrologist, Centre for Kidney Research, Children's Hospital at Westmead; and Coordinating Editor, Cochrane Renal Group, NSW.
Les M Irwig, PhD, FFPHM, Professor of Epidemiology.
Martin R Stockler, MSc(ClinEpi), FRACP, Senior Lecturer; also at Department of Medicine, and NHMRC Clinical Trials Centre, University of Sydney; and Consultant Medical Oncologist, Sydney Cancer Centre, Royal Prince Alfred Hospital and Concord Repatriation General Hospital, NSW.

Reprints will not be available from the authors.
Correspondence: Dr J C Craig, Department of Public Health and Community Medicine, Edward Ford Building A27, University of Sydney, NSW 2006.
joncAThealth.usyd.edu.au

©MJA 2001
Make a comment