Disseminating and applying best evidence

Does use of systematic reviews equal evidence-based medicine?

MJA 1998; 168: 260-261  

There are two purposes to medical research: one abstract -- the pursuit of knowledge for its own sake -- the other practical -- to improve health outcomes. To achieve the latter, research results must be disseminated and implemented, but this process may be slow, inaccurate and incomplete, resulting in varying clinical practice and outcomes. This is not through wilful disregard of medical advances; no active clinician expects to practise without needing new information and skills as health advances occur. Yet it is not humanly possible to keep up with all advances in all areas of medical research. Medical practitioners may use different forms of evidence to enlighten decisions but, until recently, were not taught how to ask the relevant questions about the new knowledge needed, how to sieve the medical literature and access information efficiently, nor how to assess success in applying new knowledge. Recognition of these deficiencies led to the discipline of evidence-based medicine (EBM). However, the recently emerged centres and departments of EBM are distrusted by some, perhaps because of visions of "cookbook" medicine, or the implied deficiencies in existing knowledge and practice, but often through misunderstanding of the nature of EBM. EBM is "a process of lifelong, self directed learning in which caring for our own patients creates the need for clinically important information about diagnosis, prognosis, therapy and other clinical and health care issues, and in which we: Convert these information needs into answerable questions; Track down with maximum efficiency the best evidence with which to answer them (whether from the clinical examination, the diagnostic laboratory, from research evidence or other sources); Critically appraise that evidence for its validity (closeness to the truth) and usefulness (clinical applicability); Apply the results of this appraisal in our clinical practice; Evaluate our performance."1 EBM is a structured process through which all available evidence is used to support clinical decision-making. Most active clinicians would support this philosophy, and data show that much clinical decision-making is in keeping with current knowledge.2-4 In this issue of the Journal, Jordens et al5 address the issue of how well Australian neonatologists and obstetricians access and use one such source of information -- systematic reviews, which aim to review all available clinical trial data in a structured way, often including meta-analyses. It is implicit in the research of Jordens et al that use of systematic reviews is "a good thing" and should be encouraged. They found that 72% of Australian neonatologists and 44% of their sample of Australian obstetricians reported using systematic reviews on average once per month. Is this a "good" result? Or should it be expressed as 28% of neonatologists and 56% of obstetricians report not using systematic reviews, with the implication that systematic reviews should be used by more? I believe neither is the issue. It remained unclear from the study whether using systematic reviews objectively improved practice, or whether respondents who used systematic reviews more also practised better, as judged by external benchmarks. Furthermore, Jordens et al found that, for keeping abreast with new clinical developments, respondents relied primarily on journals, followed (in decreasing order) by conferences and meetings, colleagues, and MEDLINE, with systematic reviews last. For clinical problem-solving, respondents relied on colleagues, MEDLINE, journals and other printed medical literature, again above systematic reviews. Does this mean that systematic reviews are not valuable? Of course not. It merely identifies them as one of many sources of information. Choice of sources will depend on the existing skills and knowledge of the individual practitioner. Although systematic reviews undoubtedly have a place in continuing medical education and may be more informative than traditional opinion-based, narrative reviews,6 we rarely acknowledge their flaws. As with any summary, they have a finite life span and must be updated obsessively and regularly. Systematic reviews on the same topic may be inconsistent.7 They are often based on meta-analyses, which may be flawed in design8 or incorrect.9 In 35% of instances, meta-analysis results are not corroborated by subsequent large scale randomised clinical trials.9 For these reasons, systematic reviews must be seen as one source of information, and not as the only or best source. They are merely a structured way of reviewing other investigators' data. Documenting and disseminating the evidence is only the beginning of continuing medical education. It does not matter how advances are disseminated, as long as it is done accurately and efficiently. More importantly, it is not the dissemination per se that is crucial, but the appropriate application and use of the information. Effective strategies for changing clinical practice include use of reminders, academic visits, opinion leaders and combinations of these.10 Other sources of information known to be important include university-sponsored continuing medical education, hospital rounds, and pharmaceutical representatives.11 Conferences and lectures are in general least effective, while colleagues and journals consistently score highly.10,11 Different strategies will work for different clinicians in different environments12 and must be adapted to the specific practice change desired, the target, setting, and obstacles to change. A series of staged alterations will usually be needed, including review of whether practice has changed and appropriate further alterations as necessary. Unfortunately, it is clear that there are still major problems with dissemination and application of evidence. For example, only 36%-42% of patients in the United States may be receiving b -blockers after myocardial infarction,13 despite their well known benefits. Systematic reviews certainly have a place in improving the dissemination and application of current best knowledge, but it is worrying when it is implied that these reviews equal EBM and that if practitioners do not use them routinely their knowledge base or clinical practice is flawed. Jordens et al have shown that systematic reviews are accessed as one source of clinical information and may alter clinical practice. However, whether or not clinicians use a particular source of information is not the important issue. The challenge is to improve further the application of best knowledge. Paddy A Phillips Professor, and Head of Medicine, Flinders University of South Australia Flinders Medical Centre, Adelaide, SA Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidence based medicine: how to practice and teach EBM. New York: Churchill Livingstone, 1997. Ellis J, Mulligan I, Rowe J, Sackett DL. Inpatient general medicine is evidence based. Lancet 1995; 346: 407-409. Geddes JR, Game D, Jenkins NE, et al. What proportion of primary psychiatric interventions are based on evidence from randomised controlled trials? Qual Health Care 1996; 5: 215-217. Gill P, Dowell AC, Neal RD, et al. Evidence based general practice: a retrospective study of interventions in one training practice. BMJ 1996; 312: 819-821. Jordens CFC, Haw P, Irwig LM, et al. Use of systematic reviews of randomised trials by Australian neonatologists and obstetricians. Med J Aust 1998; 168: 267-270. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 1997; 126: 376-380. Jadad AR, Cook DJ, Browman GP. A guide to interpreting discordant systematic reviews. Can Med Assoc J 1997; 156: 1411-1416. Bailar JC. The promise and problems of meta-analysis. N Engl J Med 1997; 337: 559-601. LeLorier J, Gregoire G, Benhaddad A, et al. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 1997; 337: 536-542. David DA, Thomson MA, Oxman AD, Haynes B. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995; 274: 700-705. Felch WC, Scanlon DM. Bridging the gap between research and practice: the role of continuing medical education. JAMA 1997; 277: 155-156. Grol R. Beliefs and evidence in changing clinical practice. BMJ 1997; 315: 418-421. Rogers WJ, Bowlby LJ, Chandra NC. Treatment of myocardial infarction in the United States (1990 to 1993): observations from the National Registry of Myocardial Infarction. Circulation 1994; 92: 2103-2114.