Connect
MJA

Do doctors know best? Comments on a failed trial

Caroline J Hunt, Louise M Shepherd and Gavin Andrews
Med J Aust 2001; 174 (3): 144-146.
Published online: 5 February 2001

A randomised controlled trial was planned to compare two different treatment strategies — structured problem solving and selective serotonin reuptake inhibitor (SSRI) medication — for patients with mild to moderate major depression. The trial was to be conducted in the primary care setting with all treatment given by general practitioners. When no patients had been recruited into the study after six months, we performed an audit of all patients with depressive symptoms attending the doctors' practices over three weeks. Exclusion criteria were changed to ease entry into the trial, but still no patients were recruited over the following six months. What went wrong?

MJA 2001; 174: 144-146

Why did the trial fail? - Acknowledgements - References - Authors' details

- - More articles on General practice and primary care


  The recent National Survey of Mental Health and Wellbeing found that depression was associated with significant disability and that 6.3% of the Australian population was estimated to have suffered a major depressive disorder in the previous 12 months.1 The survey also showed that general practices were the main points of contact for patients with a mental disorder, consistent with previous reports that only 5% of such patients are referred to psychiatrists.2 As most depressed patients will be treated in primary care, the evaluation of treatment in this setting is important.

Structured problem solving is emerging as an effective treatment for clinical depression.3-6 This treatment aims to teach patients to use their own resources to deal with their problems and includes skills such as identifying and simplifying problems, "brainstorming" potential solutions, and implementing these solutions. The treatment is brief, has clearly identified stages, and is very suitable for delivery by primary healthcare professionals. However, little research in primary care settings has used general practitioners as the main treatment providers, and thus the degree to which the evidence for the effectiveness of structured problem solving can be generalised to general practice is questionable. For this reason we designed a randomised trial with all treatment conducted by GPs (Box 1). The study was, in part, a replication of an earlier United Kingdom project that showed that problem solving was better than placebo, but equivalent to amitriptyline.4

Six months after the trial commenced no patients had been recruited.



Why did the trial fail?

In practical terms, the trial should not have failed, as mild to moderate depression is a common presentation in primary care and the research protocol addressed many of the problems identified in previous primary care research failures.9,10 For example:

  • We used GPs who had participated in a Masters program designed to improve the recognition and management of mental disorders in general practice;

  • We involved the participating GPs in the development of the protocols;

  • We made every effort to minimise tasks involved in the study for the GPs; and

  • The chief investigator was responsible for the teaching on the Masters program, and had developed a close working relationship with the GPs.

While our original intention was not to address the complexity of conducting randomised trials in clinical practice, the apparent unease of many of the GPs with randomisation raises some potential reasons for the failure (Box 2). Apparent ambivalence towards randomisation in medicine, despite strong support in the scientific literature, has been reported previously.11 Silverman argues that as medicine shifts from the traditional authoritarian stance of "doctor knows best" towards the use of clinical guidelines and standardised treatment protocols, there is an increased discomfort in any approach that might be seen to admit a lack of crucial knowledge on the part of the doctor.11 In other words, asking patients to consent to randomisation between two conditions admits an uncertainty that compromises the traditional doctor-patient relationship.

However, it has not been difficult to engage doctors in trials involving randomisation of their patients, and there is evidence that many are prepared to follow simple randomisation protocols.12 For example, one of the first randomised controlled trials of giving aspirin to patients very early after myocardial infarction (to stop or reverse the thrombotic process) enrolled 2500 GPs. These GPs, who were blind to the treatment condition they were offering, agreed to give the allocated capsules (aspirin or placebo) to any patient who presented with chest pain or other symptoms likely to be caused by a myocardial infarct. Two thousand patients were recruited into the trial, suggesting that, in this example, randomisation was not a substantial concern. If it is not randomisation per se that causes reluctance to recruit, then other factors need consideration.

In our study, the audit results indicated that one in six patients had been excluded because the GPs lacked confidence that structured problem solving would be of benefit. Perhaps the GPs lacked confidence in their ability to deliver the psychological treatment effectively. If this is the case, the failure of this trial may have been influenced by the inclusion of a psychological treatment. We have argued elsewhere that doctors remain cautious about using non-drug treatments, partly because of the lack of organised promotion of non-proprietary treatments, and partly because of the difficulty in ensuring quality control.13 Yet, these GPs had agreed problem solving was a useful treatment, had demonstrated competence as part of their training, and two GPs in the group had conducted their own research projects where they taught structured problem solving to other doctors.

It is also possible that, given the obvious differences between the psychological and pharmacological approaches, the GPs had formed an opinion early in the recruitment process that one or other approach would better suit a particular patient. In this case, exclusion from the trial was based on the presumption that the doctor already knew what treatment was best.11,14 In a disturbing example of "doctor knows best", two-thirds of suitable patients were not enrolled in a randomised trial of antiarrhythmia drugs because their doctors were so convinced of the benefits of the drugs that they did not want their patients allocated to the placebo group.15 The study eventually showed that these drugs were capable of causing fatal arrhythmias, so those doctors were essentially withholding from their patients the 50% chance of being allocated to the safer placebo alternative.

Other factors may cause doctors to hesitate when faced with recruiting patients, including the complexities of obtaining informed consent, the need to complete clinical ratings or ask patients to complete self-report questionnaires for measurement of outcome, or the need to work within a specific treatment protocol. Although tasks for the GPs were minimal, it is likely that the necessary role change from practitioner to scientist-practitioner was too great to facilitate a shift in treating behaviour. If this is the case, the inherent contradiction between research and delivery of care in the minds of many clinicians is a basic problem for clinical research.16

Nevertheless, treatments that are efficacious in research settings need to be evaluated under the conditions of routine care, and it may not be sufficient to rely on the use of qualitative methods to evaluate outcome in primary care.17 Perhaps an explicit use of the "uncertainty principle" in randomised controlled trials within routine clinical care will enhance recruitment rates.18 That is, if there is apparent certainty about what treatment is best, it is ethically untenable that patients should have their treatment chosen at random, so only patients for whom there is uncertainty about which treatment would be best should be recruited into a randomised trial. In this way the ethical dilemma for clinicians is solved, and the heterogeneity of the patient sample maintained, as clinicians will differ significantly in the types of patients they will be uncertain about. Financial incentives might increase the involvement of clinicians in research, but have caused public outcry in the United States;19,20 the resulting assertive recruiting can also erode informed consent.21,22

We argue instead that a more fundamental shift in ethos and knowledge of principles that underlie research in clinical practice is required. This argument has parallels in the recent Royal Australian College of General Practitioners report on an implementation strategy for evidence-based clinical practice guidelines.23 The report points to a lack of understanding of the principles underpinning the use of clinical practice guidelines in general practice, and the need to develop a culture in which such guidelines are used and valued. In regard to outcome research, until doctors accept a scientist-practitioner model of practice it is unlikely that they will feel comfortable using conventional research protocols.

Clinical research conducted in routine care will help clinicians make informed decisions about what may be the best treatment for their patients based on scientifically derived knowledge. We hope that the questions raised in this article will stimulate debate and research that will directly address this important issue.


Acknowledgements

This research was supported by a grant from the School of Psychiatry, University of New South Wales.


References

  1. Andrews G, Henderson S, Hall W. Prevalence, comorbidity, disability and service utilisation; and overview of the Australian national mental health survey. Br J Psychiatry 2001. In press.
  2. Gath D, Catalan J. The treatment of emotional disorders in general practice: psychological methods versus medication. J Psychosom Res 1986; 30: 381-386.
  3. Mynors-Wallis L, Davies I, Gray A, et al. A randomised controlled trial and cost analysis of problem-solving treatment for emotional disorders given by community nurses in primary care. Br J Psychiatry 1997; 170: 113-119.
  4. Mynors-Wallis LM, Gath DH, Lloyd-Thomas AR, Tomlinson D. Randomised controlled trial comparing problem-solving treatment with amitriptyline and placebo for major depression in primary care. BMJ 1995; 310: 441-445.
  5. Schulberg HC, Block MR, Madonia MJ, et al. Treating major depression in primary care practice. Eight-month clinical outcomes. Arch Gen Psychiatry 1996; 53: 913-919.
  6. Catalan J, Gath DH, Anastasiades P, et al. Evaluation of a brief psychological treatment for emotional disorders in primary care. Psychol Med 1991; 21: 1013-1018.
  7. Depression Guideline Panel. Depression in primary care. Vol. 2. Treatment of major depression. Clinical Practice Guideline No. 5. Rockville, MD: Department of Health and Human Services, Public Health Service, Agency for Health Care Policy and Research, April 1993. (AHCPR Publication No. 93-0551.)
  8. ICD-10 classification of mental and behavioural disorders. Geneva: World Health Organization, 1992.
  9. Foy R, Parry J, McAvoy B. Clinical trials in primary care. BMJ 1998; 317: 1168-1169.
  10. Peto V, Coulter A, Bond A. Factors affecting general practitioners' recruitment of patients into a prospective study. Family Practice 1993; 10: 207-211.
  11. Silverman W. Equitable distribution of the risks and benefits associated with medical innovations. In: Maynard A, Chalmers I, editors. Non-random reflections on health services research: on the 25th anniversary of Archie Cochrane's effectiveness and efficiency. London: BMJ Publishing Group, 1997: 184-193.
  12. Elwood P. Cochrane and the benefits of aspirin. In: Maynard A, Chalmers I, editors. Non-random reflections on health services research: on the 25th anniversary of Archie Cochrane's effectiveness and efficiency. London: BMJ Publishing Group, 1997: 107-121.
  13. Andrews G. On the promotion of non-drug treatments. BMJ 1984; 289: 994-995.
  14. Segelov E, Tattersall MHN, Coates AS. Redressing the balance -- the ethics of not entering an eligible patient on a randomised trial. Ann Oncol 1992; 3: 103-105.
  15. Moore TJ. Deadly medicine. New York: Simon and Schuster, 1995.
  16. Tognoni G, Alli F, Avanzini F, et al. Randomised clinical trials in general practice: lessons from a failure. BMJ 1991; 303: 969-971.
  17. Miller ML, Crabtree BF. Qualitative analysis: how to begin making sense. Family Practice Res J 1994; 14: 289-297.
  18. Collins R, Peto R, Gray R, Parish S. Large-scale randomised evidence: trials and overviews. In: Maynard A, Chalmers I, editors. Non-random reflections on health services research: on the 25th anniversary of Archie Cochrane's effectiveness and efficiency. London: BMJ Publishing Group, 1997: 197-230.
  19. Eichenwald K, Kolata G. Drug trials hide conflicts for doctors. New York Times, May 16, 1999; 1,28-29.
  20. Eichenwald K, Kolata G. A doctor's drug trials turn into fraud. New York Times, May 17, 1999.
  21. Ferguson C. Payment of financial incentives to GPs may invalidate informed consent process. BMJ 1998; 316: 75-76.
  22. Shalala D. Protecting research subjects -- what must be done. N Engl J Med 2000; 343: 808-810.
  23. Report on consultancy to develop an implementation strategy for CPGs. Sydney: Royal Australian College of General Practitioners, August 2, 2000.



Authors' details

School of Psychiatry, University of New South Wales, Sydney, NSW.
Caroline J Hunt, MPsych, PhD, Lecturer (currently, Senior Lecturer, Department of Psychology, University of Sydney).
Gavin Andrews, MD, Professor.

Clinical Research Unit for Anxiety Disorders, St Vincent's Hospital, Sydney, NSW.
Louise M Shepherd, BA(Hons), MPsych, Clinical Psychologist.
Reprints will not be available from the authors.
Correspondence: Dr C J Hunt, Department of Psychology (F12), University of Sydney, NSW, 2006.
carolineATpsych.usyd.edu.au


Make a comment





1: The planned trial

Objective: To compare structured problem solving with SSRI medication and non-specific counselling for mild to moderate major depression.

Trial development: The trial was initially designed with three treatment arms: structured problem solving, medication and placebo. In 1997, general practitioners in their second year of a two-year part-time Master of Psychological Medicine course were approached for feedback. The GPs indicated that, while they believed the study to be of value, they were uncomfortable with the use of a placebo control, so the placebo arm of the trial was abandoned. Detailed protocols for each treatment were developed, with medications which reflected best prescribing practice (United States Department of Health and Human Services Clinical practice guidelines 7), a treatment period of six months, and a follow-up period to track longer-term changes. The protocols were designed to be simple to use and, to minimise tasks for the GPs, telephone assessments by a clinical psychologist were planned to confirm diagnosis, assess severity, and evaluate outcome.

Design: Patients assessed by their GPs as having mild to moderate major depression were to be randomly allocated to:

  • structured problem solving alone;
  • selective serotonin reuptake inhibitor (SSRI) medication and non-specific counselling; or
  • SSRI medication and structured problem solving.

Patients were required to meet International classification of diseases - 10th revision 8 criteria for a mild or moderate major depressive episode. Exclusion criteria included current or previous manic (or hypomanic) or psychotic symptoms, current suicidal intent, current drug or alcohol abuse, current pharmacological or psychological treatment for depression, and (to exclude severe depression) current somatic (or melancholic) features.

Participating general practitioners: In 1998, the protocols were again reviewed by students in the Masters course, who agreed that they were consistent with current best primary care practice and could be delivered in this setting. Ten GPs agreed to participate - four were current second-year students in the Masters course, and six were graduates of the course. Three current second-year students did not participate because they were not working in general practice, and two because they did not wish to randomly allocate their patients to treatment. All participating doctors had been trained in assessing and managing depression (including structured problem solving), and clinical supervision was offered over the course of the trial.

Back to text
 

2: Why were no patients recruited?

Six months after commencement of the trial in 1998, despite frequent reminders and discussions about the trial protocol, no patients had been recruited. This prompted a clinical audit of all patients with depressive symptoms attending the participating doctors' practices over three weeks. Each general practitioner recorded patients presenting with depressive symptoms and the reasons why they considered them unsuitable for the trial on a form we designed for this purpose. Over the three weeks, 114 patients presented with depressive symptoms (12% of all presenting patients), but none were entered in the study. The results of the audit are shown in the Table. The most frequently cited reasons for exclusion were current pharmacological or psychological treatment for depression, or depression of insufficient severity to meet ICD-10 criteria. In an attempt to improve recruitment in the following six months, we dropped the exclusion criteria of current psychological treatment and insufficient severity (these features now to be assessed by the clinical psychologist), yet still no patients were recruited.

Reasons for not entering trial No. (%) patients*

Depression insufficiently severe to meet ICD-10 criteria 27 (23.6%)
Severe depression or criteria met for "somatic syndrome" 13 (11.4%)
Prior or current manic, hypomanic or psychotic episode 6 (5.3%)
Serious suicidal intent 5 (4.4%)
Current drug or alcohol abuse 6 (5.3%)
Current pharmacological treatment for depression 38 (33.3%)
Current psychological treatment for depression 34 (29.8%)
Physical problems precluding use of an SSRI 1 (0.9%)
Prior failure to respond to an SSRI 5 (4.4%)
Patient refused randomisation 7 (6.1%)
General practitioner not confident about using problem solving with this patient 20 (17.5%)
Other (eg, dementia, communication difficulties, personality disorders) 22 (19.3%)

*There were 114 patients, but general practitioners frequently nominated more than one reason for excluding patients. ICD-10=International classification of diseases - 10th revision.8 SSRI=selective serotonin reuptake inhibitor.
Once the trial was formally abandoned, there were discussions with the four GPs still completing their Masters course. These GPs admitted unease with the process of randomisation, despite the demonstrated efficacy of both treatments for this population.
Back to text

Received 13 July 2020, accepted 13 July 2020

  • Caroline J Hunt
  • Louise M Shepherd
  • Gavin Andrews


Correspondence: 

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Online responses are no longer available. Please refer to our instructions for authors page for more information.