Connect
MJA
MJA

Use of systematic reviews of randomised trials by Australian neonatologists and obstetricians

Penelope Hawe, Les M Irwig, David J Henderson-Smart, Margaret Ryan and Deborah A Donoghue
Med J Aust 1998; 168 (6): 267-270.
Published online: 16 March 1998

Use of systematic reviews of randomised trials by Australian neonatologists and obstetricians

Christopher F C Jordens, Penelope Hawe, Les M Irwig, David J Henderson-Smart, Margaret Ryan, Deborah A Donoghue, Roger G Gabb and Ian S Fraser

MJA 1998; 168: 267-270
For editorial comment see Phillips
 

Abstract - Introduction - Methods - Results - Discussion - Acknowledgements - References - Authors' details

- - ©MJA1998
 

Abstract

Objective: To determine what proportion of Australian neonatologists and obstetricians report using systematic reviews of randomised trials.
Design: Cross-sectional survey using structured telephone interviews.
Setting: Australian clinical practice in 1995.
Participants: 103 of the 104 neonatologists in Australia (defined as clinicians holding a position in a neonatal intensive care unit); a random sample of 145 members of the Royal Australian College of Obstetricians and Gynaecologists currently practising in Australia.
Main outcome measures: Information sources used in clinical practice; reported awareness of, access to and use of systematic reviews, and consequent practice changes.
Results: Response rates were 95% (neonatologists) and 87% (obstetricians); 71 neonatologists (72%) and 55 obstetricians (44%) reported using systematic reviews, primarily for individual patient care. Databases of systematic reviews were used with a median frequency of once per month. Among neonatologists, systematic reviews were used more commonly by those who were familiar with computers, attended professional meetings, and had authored research papers. Among obstetricians, they were used more commonly by those who were familiar with computers, had less than 10 years' clinical experience, attended more deliveries, and were full-time staff specialists in public hospitals. Of neonatologists who reported using systematic reviews, 58% attributed some practice change to this use. For obstetricians, the corresponding figure was 80%.
Conclusions: There is evidence that Australian neonatologists and obstetricians use systematic reviews and modify their practice accordingly. Dissemination efforts can benefit from knowledge of factors that predict use of systematic reviews.  

Introduction

Randomised controlled trials are widely accepted as the best method of evaluating the effectiveness of medical interventions. However, their findings are slow to change medical opinion and practice.1,2 The communication of clinically important research findings is hampered by the volume and geometric growth of the medical literature.3

Reviews address this problem, but conventional review methodology is unscientific4,5 and open to selection biases inherent in the publication process.5,6 This has led to the development of systematic reviews. A properly conducted systematic review begins with an exhaustive search for published and unpublished randomised trials addressing a well defined research question. The second step is to identify trials of adequate quality to contribute to decision-making. The results of included trials are then pooled, or "meta-analysed", to arrive at a quantitative estimate of the benefits and harms of treatment. Systematic reviews published by the Cochrane Collaboration are also continuously updated by specialist review groups.7

Systematic reviews in pregnancy and childbirth have been available for some years. In 1988 a comprehensive trial register became commercially available in database form as the Oxford database of perinatal trials.8 Effective care in pregnancy and childbirth, 9 a textbook based on this research synthesis, was published the following year, with a guide for non- specialist readers.10 A companion textbook, Effective care of the newborn infant,11 appeared in 1992. From 1993, systematic reviews relating to pregnancy and childbirth were available electronically in the Cochrane pregnancy and childbirth database.12 All these reviews are currently being updated for The Cochrane library.13

These new publications represent prototypes for future publications in other fields of practice. However, their impact on clinical practice remains uncertain, despite several surveys to ascertain the extent of their dissemination and uptake.14-18 To make a practical difference, systematic reviews must be readily available to clinicians who are aware of them, who use them, and who implement their findings. To ascertain whether this is occurring in Australia, we conducted a survey of neonatologists and obstetricians with the aims:

  • To gauge awareness of, access to, and use of the Oxford database, Cochrane database, and Effective care textbooks;

  • To place these resources in the context of other information sources that inform practice in obstetrics and neonatology; and

  • To ascertain how often and why these resources are consulted, predictors of their use, and whether their use has led to reported changes in clinical practice.
 

Methods

 

Participants

Clinicians who held a position in one of Australia's 23 neonatal intensive care units were eligible for the survey. The Australian and New Zealand Neonatal Network (ANZNN), which covers all neonatologists, provided a current listing. The Royal Australian College of Obstetricians and Gynaecologists (RACOG) drew a simple random sample of 20% of Fellows from its membership. Those sampled were eligible if they were currently practising obstetrics. Any clinicians not currently practising in Australia, not on the telephone network, or who were involved in designing this survey, were ineligible.

Eligible clinicians were sent an introductory letter, telephoned, and invited to schedule a confidential, 10-minute telephone interview about information sources used in clinical practice. Systematic reviews were not mentioned until the interview was under way.  

Interviews

An existing interview schedule15,16 was modified in consultation with the ANZNN and the RACOG. Respondents were first asked to name the three main sources of information they used for keeping up with new developments in their field, and for dealing with uncertainty about a specific treatment decision. They were also asked which of the three information sources they considered the most important or useful for each task.

Respondents were then asked about their access to and use of computers, and asked directly whether they had heard of, had access to, and used the Cochrane database, Oxford database, and Effective care of the newborn infant (neonatologists) or Effective care in pregnancy and childbirth (obstetricians).

Those who reported using any of these resources were asked what a systematic review was, how often they consulted that resource, what they used it for, whether they thought using it had made any difference to their clinical practice and, if so, whether they could name a treatment policy they had altered in response to evidence from a systematic review. To count as users of systematic reviews, respondents had to mention randomised trials or meta-analysis in their description of a systematic review.

The final questions sought demographic information on clinical experience, place of education, attendance at professional meetings, research publications, academic appointments, and qualifications. A research degree was defined as a doctorate, relevant master's degree, or bachelor's degree of science in medicine, held in addition to basic medical and specialty qualifications. Reports of authorship were verified by searching MEDLINE and by screening abstracts.

For neonatologists it was noted whether they held full- or part-time positions, whether they worked in a hospital with an obstetric unit, whether they headed a neonatal unit, and in which unit they worked. For obstetricians, it was noted whether they held a position as a full-time staff specialist in a public hospital. They were asked how many deliveries they attended each year, and whether they worked at any hospital with an accredited RACOG training post.

The interview schedule was modified after a pilot study with seven neonatologists in New Zealand. The main survey was conducted between August and December 1995, with ethics approval from the ANZNN, RACOG and the University of Sydney. All interviews were conducted by the first author (C F C J).  

Analysis

Confidence intervals for the proportion of obstetricians who reported using systematic reviews were calculated with a finite population correction. Confidence intervals were not calculated for the corresponding proportion of neonatologists, as this was ascertained for all Australian neonatologists. Descriptive and c 2 analyses were conducted. Then, using a backwards stepwise modelling procedure, a multivariate logistic regression analysis was used to determine which study factors predicted the reported use of systematic reviews. With the neonatology data, this modelling procedure was repeated using binomial generalised estimating equations to adjust for clustering of respondents within neonatal units.19  

Results

Of 106 listed neonatologists, three were ineligible (one had retired, one was not practising in Australia, and one was involved in designing this survey). Of the 103 eligible clinicians remaining, 98 completed interviews (95% response). From a sample of 210 obstetricians, 65 were ineligible for the survey (51 no longer practised obstetrics, 13 were not practising in Australia, and one could not be located on the telephone network). Of 145 eligible obstetricians, 126 completed interviews (87% response). Respondent characteristics are shown in Box 1, with comparative data on computer use.


For the purpose of keeping up with new clinical developments, respondents favoured journals, conferences and meetings, colleagues and MEDLINE (in that order) over systematic reviews. For the purpose of clinical problem-solving, colleagues, MEDLINE, journals, and other printed medical literature were favoured over systematic reviews as sources of information. Although the percentages favouring each source varied between the two specialties and according to whether the respondent was simply nominating the resource or nominating it as the most important resource, the rank order of the resources remained consistent.

In each specialty, 21% of respondents mentioned systematic reviews without prompting from the interviewer when naming the information sources they used, or when describing their computer use. When questioned directly about systematic reviews, 71 neonatologists (72%) and 55 obstetricians (44%; 95% confidence interval, 36%-51%) reported using them in either electronic database or textbook form. The confidence interval indicates the reliability of the sample prevalence as an estimate of the population prevalence. The Figure shows reported levels of awareness of, access to and use of systematic reviews by specialty.


 

Predictors of
use of systematic
reviews

Results of the multivariate analysis are shown in Box 2. Among neonatologists, three factors significantly (P<0.05) and independently predicted use of systematic reviews: attendance at meetings of the Australian Perinatal Society, authorship of at least one research paper, and familiarity with computers (our index of this was use of a computer for word-processing). Results from cluster analysis agreed with those from logistic regression.

Among obstetricians, four factors sig nificantly and independently predicted use of systematic reviews: familiarity with computers, mode of practice (full-time staff specialists were more likely to use systematic reviews than others), clinical experience (recent graduates were more likely to use systematic reviews), and number of deliveries per year (likelihood of using systematic reviews increased with this number).  

Descriptors of
use of systematic
reviews

Both neonatologists and obstetricians reported using databases of systematic reviews with a median frequency of once per month. The main purpose for using systematic reviews (irrespective of their format) reported by respondents from both specialties was individual patient care. Systematic reviews were also said to be used for (in rank order of frequency of reporting) teaching, preparing a pre sentation, reviewing current management or developing evidence-based protocols or guidelines, settling disputes, background information, reviewing a topic, research, as a source of references, and patient information.

Of the 71 neonatologists who reported using systematic reviews, 58% said that this had changed their clinical practice in some way, and 44% gave at least one example of a treatment policy they had altered in response to a systematic review. The most common examples were treatment of respiratory disorders, and use of steroids and indomethacin. Corresponding percentages for obstetricians who reported using systematic reviews were 80% (reported a change in practice) and 71% (gave at least one example). The most common example was use of steroids in management of preterm rupture of membranes.  

Discussion

We found that 72% of neonatologists and 44% of obstetricians reported consulting systematic reviews, primarily for the purpose of individual patient care. Databases of systematic reviews were used at a median frequency of once per month. Systematic reviews were used more commonly by those who were familiar with computers in both specialties, by those who attended professional meetings and had authored research papers among neonatologists, and by those who had had less than 10 years' clinical experience, attended more deliveries, and who had a position as a full-time staff specialist among obstetricians.

The findings of this survey are based on self-reporting. Although the survey was designed to minimise over-reporting, it was still subject to imprecision in respondents' recall about (for example) the frequency with which they used systematic reviews, and to difficulties in attributing practice changes to their use. Nevertheless, this was the first study of the use of systematic reviews by Australian clinicians, and the findings have immediate and practical relevance for organisations such as universities, government agencies and specialty colleges that are trying to improve access to the best available evidence and to promote its use.20-23

The association between use of systematic reviews and attendance at Australian Perinatal Society meetings among neonatologists could reflect greater receptiveness to innovations among clinicians who attend professional meetings. However, it also supports the perception (reported elsewhere13 ) that professional organisations play an important role in disseminating research findings. Future dissemination efforts could usefully concentrate on these organisations.

In both specialties, familiarity with computers predicted use of systematic reviews. Better access to digital information technology and training in its use is therefore likely to enhance uptake of these reviews. This might include improved access to The Cochrane library and the Internet, and training workshops for searching specialist databases. The remaining predictors of use of systematic reviews, as well as the overall contrast between neonatology and obstetrics, suggest that specialists practising primarily in public hospitals are more likely to use systematic reviews than those working primarily in private practice. Special efforts are therefore needed to reach the latter. As systematic reviews do not appear to rank highly among the information sources used in clinical practice, it is important to stress their relative advantages. Uptake of this innovation appears more likely among recent graduates, with senior clinician researchers acting as "product champions".24

Although debate persists as to whether practising clinicians accept the innovations of evidence-based medicine,25 our survey found evidence that Australian neonatologists and obstetricians use systematic reviews, and appear to modify their practice accordingly. Efforts are needed to enhance the use of these reviews, and to conduct further evaluations of their influence on clinical practice. In attempting to improve practice standards, it is important to pursue methods which have been shown to be effective.26  

Acknowledgements

We would like to thank the many clinicians who gave up their time to participate in this survey. Infrastructure support for this project was provided by the Department of Public Health and Community Medicine at the University of Sydney. Special thanks are due to Petra Macaskill, who conducted the cluster analysis, and Jeanette Ward and Mary Osborne, from the Central Sydney Area Health Service Needs Assessment and Health Outcomes Unit, who provided helpful advice on survey procedures.  

References

  1. Antman EM, Lao J, Kupelnick B, et al. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts: treatments for myocardial infarction. JAMA 1992; 268: 240-248.
  2. Stross JK, Harlan WR. The dissemination of new medical information. JAMA 1979; 241: 2622-2624.
  3. Warren KS. From papyrus to parchment to paper to pixels: information technology and the future of biomedical publishing. In: Lock S, editor. The future of medical journals. London: BMJ, 1991: 127-146.
  4. Mulrow CD. The medical review article: state of the science. Ann Intern Med 1987; 106: 485-488.
  5. Light RJ, Pillemer DB. Summing up: the science of reviewing research. Cambridge: Harvard University Press, 1984.
  6. Dickersin K, Berlin JA. Meta-analysis: state-of-the-science. Epidemiol Rev 1992; 14: 154-176.
  7. Sackett DL. The Cochrane Collaboration. ACP J Club 1994; 120 Suppl 3: A-11.
  8. Chalmers I, editor. Oxford database of perinatal trials. Version 1.2, disk issue 7 (Spring). Oxford: Oxford University Press, 1992.
  9. Chalmers I, Enkin M, Keirse MJNC. Effective care in pregnancy and childbirth. Oxford: Oxford University Press, 1989.
  10. Enkin M, Keirse MJNC, Chalmers I. A guide to effective care in pregnancy and childbirth. Oxford: Oxford University Press, 1989.
  11. Sinclair JC, Bracken MB. Effective care of the newborn infant. Oxford: Oxford University Press, 1992.
  12. Enkin MW, Keirse MJNC, Renfrew MJ, Neilson JP, editors. Cochrane pregnancy and childbirth database. Oxford: Update Software, 1993.
  13. The Cochrane Library [database on disk and CD-ROM]. The Cochrane Collaboration. Oxford: Update Software; 1996. Updated quarterly.
  14. Lomas J. Retailing research: increasing the role of evidence in clinical services for childbirth. Milbank Q 1993; 71: 439-475.
  15. Stocking B. Implementing the findings of effective care in pregnancy and childbirth in the United Kingdom. Milbank Q 1993; 71: 497-523.
  16. Paterson-Brown S, Fisk NM, Wyatt JC. Uptake of meta-analytical overviews of effective care in English obstetric units. Br J Obstet Gynaecol 1995; 102: 297-301.
  17. Paterson-Brown S, Wyatt JC, Fisk NM. Are clinicians interested in up to date reviews of effective care? BMJ 1993; 307: 1464.
  18. Hyde C. Who uses the Cochrane pregnancy and childbirth database? BMJ 1995; 310: 1140-1141.
  19. Zeger SL, Liang K-Y. Longtitudinal data analysis for discrete and continuous outcomes. Biometrics 1986; 42: 121-130.
  20. Rychetnik L. Evidence-based medicine in the GMP (Graduate Medical Program). Focus: graduate medical degree news and events. Sydney: 1995: 1-2.
  21. Liddle J, Williamson M, Irwig L. Method for evaluating research and guideline evidence. Sydney: NSW Department of Health, 1996.
  22. National Health and Medical Research Council. Clinical practice guidelines: the management of early breast cancer. Canberra: NHMRC, 1995.
  23. Commonwealth Department of Human Services and Health. Guidelines for the development and implementation of clinical practice guidelines. Canberra: the Department, 1995.
  24. Rogers EM. Diffusion of innovations. New York: MacMillan, 1983.
  25. Miles A, Bentley P, Polychronis A, Grey J. Evidence-based medicine: why all the fuss? J Evaluation Clin Pract 1997; 2: 83-85.
  26. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 1993; 342: 1317-1322.

(Received 11 Jun, accepted 11 Sep, 1997)  


Authors' details

University of Sydney, Sydney, NSW.
Christopher F C Jordens, MPH, Postgraduate Student, Department of Public Health and Community Medicine; now Researcher, Centre for Values, Ethics and the Law in Medicine, Department of Surgery, University of Sydney;
Penelope Hawe, MPH, Senior Lecturer, Department of Public Health and Community Medicine;
Les M Irwig,
FFPHM, PhD, Associate Professor, Department of Public Health and Community Medicine;
David J Henderson-Smart,
PhD, FRACP, Professor, and Director, NSW Centre for Perinatal Health Services Research, and Department of Neonatal Medicine, Royal Prince Alfred Hospital, Sydney, NSW;
Deborah A Donoghue,
RN, BSocSc, Senior Research Assistant, Australian Institute of Health and Welfare National Perinatal Statistics Unit;
Ian S Fraser,
MD, FRACOG, Professor in Reproductive Medicine, Department of Obstetrics and Gynaecology, Queen Elizabeth II Research Institute for Mothers and Infants.

Royal Australian College of Obstetricians and Gynaecologists, Melbourne, VIC.
Margaret Ryan, MSW, PhD, Research Officer.

Centre for Professional Development, Victoria University of Technology, Melbourne, VIC.
Roger G Gabb, PhD, Professor, and Director.

Reprints: Mr C F C Jordens, Department of Surgery, Blackburn Building D06, University of Sydney, NSW 2006.
E-mail: cjordens AT surgery.usyd.edu.au


Readers may print a single copy for personal use. No further reproduction or distribution of the articles should proceed without the permission of the publisher. For permission, contact the Australasian Medical Publishing Company
Journalists are welcome to write news stories based on what they read here, but should acknowledge their source as "an article published on the Internet by The Medical Journal of Australia <http://www.mja.com.au>".

<URL: http://www.mja.com.au/>

Received 19 June 2018, accepted 19 June 2018

  • Penelope Hawe
  • Les M Irwig
  • David J Henderson-Smart
  • Margaret Ryan
  • Deborah A Donoghue


Correspondence: 

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Responses are now closed for this article.