Medicine and the community

Volume 172 - Issue 1

Patient attitudes to commonly promoted medical interventions

Authors:  Stephen P Fitzgerald and George Phillipov

Med J Aust 2000; 172 (1): 9-12.
Published online: 3 January 2000
Medicine and the community

Patient attitudes to commonly promoted medical interventions

Stephen P Fitzgerald and George Phillipov

MJA 2000; 172: 9-12
For editorial comment, see Bastian

Introduction - Methods - Data sources - Questionnaire - Design and settings - Statistical analysis - Results - Discussion - References - Authors' details
- - More articles on Social issues


Abstract Objective: To survey attitudes about three "best practice" medical interventions (hormone replacement therapy [HRT], thrombolysis for acute myocardial infarction [THROM] and coronary artery by-pass surgery [CABS]) in a sample of patients, and identify factors associated with those attitudes.
Settings: Metropolitan tertiary care hospital outpatient clinics (survey 1, April 1997), two general practice surgeries (survey 2, May 1997), and one general practice surgery (survey 3, October 1997).
Design: Patients completed a questionnaire while waiting for their clinical consultation. Attitude scores were measured on an 11-category Likert scale ranging from - 5 (definitely would not) to + 5 (definitely would) for acceptance of proposed medication or surgery.
Participants: 85 (participation rate, 85%), 77 (94%) and 95 (97%) in surveys 1, 2 and 3, respectively. Surveys 1 and 2 constituted the primary study group (n = 162). Patients aged ≥50 years or reporting heart disease were excluded from the HRT analyses; patients aged ≥65 years were excluded from the THROM and CABS analyses.
Results: The median attitude scores for HRT (n = 58), THROM and CABS (n = 111) were - 2.95 (95% CI, - 5 to - 2.1), - 0.5 (95% CI, - 0.9 to 0) and - 0.1 (95% CI, - 0.5 to + 1.3), respectively. Decreasing the risk-benefit ratio fourfold for HRT in survey 3 (n = 68) increased the median score to - 0.75 (95% CI, - 2.3 to 0).
Conclusions: Patients do not view favourably the risk-benefit ratio of the three surveyed medical interventions. These attitudes may present a major impediment to most primary prevention programs.


Introduction In recent years, evidence-based medicine has been proposed as a fundamental advance in the delivery of "best practice'' medicine.1 Accordingly, an implicit requirement exists for physicians to be aware of the best evidence, which has been suggested to be the outcomes of randomised clinical trials or the meta-analyses of such trials.2

However, consensus has also developed that patient empowerment, through complete specification of disease and treatment risks as well as likely outcome benefits, should be standard in any decision-making process.3 While evidence-based medicine may lead to a clear preference by doctors for treatment, acceptance of this same preference by patients, when apprised of all the relevant risks and benefits, has, to our knowledge, received limited study.4,5 Moreover, Schwartz et al6 have presented findings which question the ability of many patients to comprehend elementary probability concepts, understanding that is crucial to any goal of patient empowerment. There are also reports which suggest that the necessity for proper, informed choice has been compromised,7 true risk odds overstated8 or biased epidemiological data presented9 to ensure patient participation. Such reports infer that patients may not necessarily perceive, or be able to perceive, the risk-benefit ratio for many treatments as favourably as their physicians.

The purpose of our study was to analyse the questionnaire-based attitudes of patients on the risks and benefits of a number of prominently promoted medical interventions.10-12


Methods

Data sources The risk-benefit data for the questionnaires were based on the published results of trials for hormone replacement therapy (HRT),13-15 thrombolysis for acute myocardial infarction (THROM),16 and coronary artery bypass surgery (CABS).17

Questionnaire The results of published trials were reported as an absolute (rather than relative) percentage of the likely benefit or adverse outcome, to reduce any potential bias with respect to patients' interpretation of results.18-21 To minimise response bias due to specific diseases, the questionnaire statements were generically worded; participants did not know the name of the therapy. Participants were asked to indicate their preference on an 11-category Likert scale ranging from "definitely would not" to "definitely would" accept the treatment (medication or surgery). For analysis, responses were coded from - 5 to + 5, with zero representing uncertainty.

The survey participants were also given an option to provide free-text comments for their responses. These were subsequently categorised into 10 general themes: the age at which intervention occurs; prior experience of the situation (either personal or acquaintances); the odds of success offered by the intervention; an aversion to taking any form of risk; a belief that life-events are predetermined; full confidence in doctors' advice; prefer alternative medicine procedures; insufficient information to make decision; a willingness to take any risk for better health; and all other reasons

Design and settings
The first survey was conducted at the Queen Elizabeth Hospital's endocrine and diabetes outpatient clinics (April 1997), and was coordinated by the nursing staff. Queen Elizabeth Hospital is a metropolitan, university-affiliated teaching hospital.

The second survey was conducted at two general practice surgeries, which each received 50 questionnaires (May 1997). Each surgery's receptionist handled patient recruitment. The survey used the same questionnaire as the first survey, but without the "no difference" effect phrase, to assess whether this phrase had had an overly negative bias on patient scores.

After preliminary assessment of the two surveys, a third was conducted (October 1997) at a single general practice surgery, to assess the effect of a marked decrease in the risk-benefit ratio on patients' attitude towards HRT.

In all three surveys, patients were approached and asked to complete a questionnaire while waiting for their clinical consultation. One hundred questionnaires were made available for each survey, and each survey was conducted over a two-week period. There were no exclusion criteria for survey recruitment. The main reason for patient refusal was poor reading skills.

The demographic and other details of participants are shown in Box 1.

As the goal of our study was to assess patient attitudes in a prospective manner, age exclusion criteria (and cardiovascular disease comorbidity for HRT) consistent with the proposed intervention were applied before analysis.

The study was approved by the hospital ethics committee.

Statistical analysis Non-parametric statistical tests were performed using SPSS 6.1.22 Confidence intervals for medians and differences were calculated as previously described.23

Results On an 11-point Likert scale, where - 5 indicates "definitely would not" and +5 indicates "definitely would" accept the treatment, the median response for HRT in surveys 1 and 2 was - 2.95 (95% CI, - 5 to - 2.1). In survey 3, with a fourfold decrease in risk-benefit, this was significantly increased to - 0.75 (95% CI, - 2.3 to 0). For THROM, the median response was - 0.5 (95% CI, - 0.9 to 0), and for CABS, - 0.1 (95% CI, - 0.5 to + 1.3). For further details, see Boxes 2 and 3.

Discussion Our results indicate that a discrepancy in perceived benefit exists between our participants and that of the medical community towards certain common medical interventions. In none of the responses to the questions posed was there a median score that signified acceptance of the proposed risk-benefit for the intervention. Our findings suggest that patients' perceptions are more influenced by the situation and risk than the odds of successful outcome.

Thus, asymptomatic patients offered primary prevention for heart disease (HRT) had a very negative attitude, but viewed THROM and CABS treatment more favourably. This may reflect that patients consider these latter situations as potentially life-threatening and are therefore more willing to seek benefit (life) as a matter of urgency (see Box 3 free-text responses). Alternatively, it might be argued that the change in score relates to the absence of cancer as a potential side-effect, given the alarm this disease engenders.24

We were able to demonstrate that lowering the risk-benefit for HRT led to a less negative attitude. Given the magnitude in risk-benefit reduction (fourfold) we consider the overall change modest, since even at this degree of risk-benefit the attitude towards HRT was still negative. We therefore believe that our findings are essentially robust with respect to moderate changes in risk-benefit.

A limitation of our study was the small sample size, which decreased the statistical power to detect differences. The external generalisability of the findings is also limited by a lack of information on participants' socioeconomic status and level of education. However, the similarity of attitude scores across the different settings infers a degree of robustness towards such confounders.

The favourable attitude towards CABS by patients with existing heart disease is consistent with a recent report that patients actively seek aggressive post-myocardial infarction treatment from their doctors.25 The difference between the findings for THROM and CABS may reflect greater public awareness and expectations concerning CABS treatment.

It may be argued that our results, which show an overall equivocal attitude towards "best-practice" guidelines, are confounded by patients' inability to adequately interpret the probabilistic concepts.6 However, about 75% of patients providing written responses to the HRT questions indicated a clear aversion to any risk-taking; this same concern, although diminished, was also described for THROM and CABS. Moreover, for THROM and CABS, while a significant proportion of patients claimed that their decision was influenced by the odds, the actual median scores suggest the odds were viewed unfavourably.

A fundamental problem with evidence-based medicine, therefore, is that it encourages practices by physicians which may often conflict with the evolving paradigm of shared decision-making and even their own value judgements with respect to the risks and benefits of certain treatments. This is most likely to occur for preventive treatments, a medical strategy least favoured by our patients. It is also important to acknowledge that a decision by a patient that is different from that advocated by their physician does not imply the patient does not understand the information.26

In conclusion, our findings show that patients have a variety of reasons for accepting or rejecting particular medical strategies, and, even where those reasons are based directly on risk-benefit outcomes, they are seldom as positive as proponents of evidence-based medicine might expect. Accordingly, widespread promulgation and instigation of "best-practice" guidelines for the medical interventions evaluated in this study would appear limited by any increasing patient role in the decision-making process.27


References
  1. Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 1992; 268: 2420-2425.
  2. National Health and Medical Research Council. A guide to the development, implementation and evaluation of clinical practice guidelines. Canberra: Commonwealth of Australia, 1999. Available at <http://www.nhmrc.health.gov.au/publicat/synopses/cp30syn.htm>.
  3. Bogardus Jr ST, Holmboe E, Jekel JF. Perils, pitfalls, and possibilities in talking about medical risk. JAMA 1999; 281: 1037-1041.
  4. McNeil BJ, Weichselbaum R, Pauker SG. Fallacy of the five-year survival in lung cancer. N Engl J Med 1978; 299: 1397-1401.
  5. McNeil BJ, Weichselbaum R, Pauker SG. Speech and survival: tradeoffs between quality and quantity of life in laryngeal cancer. N Engl J Med 1981; 305: 982-987.
  6. Schwartz LM, Woloshin S, Black WC, Welch HG. The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med 1997; 127: 966-972.
  7. Searle J. Routine antenatal screening: not a case of informed choice. Aust N Z J Pub Health 1997; 21: 268-274.
  8. Marteau TM, Kidd J, Cook R, et al. Perceived risk not actual risk predicts uptake of amniocentesis. Br J Obstet Gynaecol 1991; 98: 282-286.
  9. Slaytor EK, Ward JE. How risks of breast cancer and benefits of screening are communicated to women: analysis of 58 pamphlets. BMJ 1998; 317: 263-264.
  10. Col NF, Eckman MH, Wong JB, Pauker SG. Strategies for individualizing patient decisions about hormone therapy. J Clin Endocrinol Metab 1999; 84: 1799-1802.
  11. Antman EM, Braunwald E. Acute myocardial infarction. In: Fauci AS, Braunwald E, Isselbacher KJ, et al, editors. Harrison's principles of internal medicine. 14th ed. New York: McGraw-Hill, 1998; 1352-1365.
  12. Loop FD. The surgical treatment of artherosclerotic coronary heart disease. In: Schlant RC, Alexander RW, editors. Hurst's The heart. 8th ed. New York: McGraw-Hill, 1994; 1367-1380.
  13. Grodstein F, Stampfer MJ, Manson JE, et al. Postmenopausal estrogen and progestin use and the risk of cardiovascular disease. N Engl J Med 1996; 335: 453-461.
  14. Falkeborn M, Persson I, Adami H-O, et al. The risk of acute myocardial infarction after oestrogen and oestrogen-progestogen replacement. Br J Obstet Gynaecol 1992; 99: 821-828.
  15. Colditz GA, Markinson SE, Hunter DJ, Willett WC. The use of oestrogens and progestins and the risk of breast cancer in post-menopausal women. N Engl J Med 1995; 332: 1589-1593.
  16. Gruppo Italiano per lo Studio della Streptochinasi nell'Infarto Miocardico (GISSI). Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet 1986; i: 397-402.
  17. The Veterans Administration Coronary Artery Bypass Surgery Cooperative Study Group. Eleven-year survival in the Veterans Administration randomised trial of coronary bypass surgery for stable angina. N Engl J Med 1984; 311: 1333-1339.
  18. Forrow L, Taylor WC, Arnold RM. Absolute relative: how research results are summarized can affect treatment decisions. Am J Med 1992; 92: 121-123.
  19. Malenka DJ, Baron JA, Johansen S, et al. The framing effect of relative and absolute risk. J Gen Intern Med 1993; 8: 543-548.
  20. Skolbekken J-A. Communicating the risk reduction achieved by cholesterol reducing drugs. BMJ 1998; 316: 1956-1958.
  21. Bunker JP, Houghton J, Baum M. Putting the risk of breast cancer in perspective. BMJ 1998; 317: 1307-1309.
  22. SPSS [computer program]. Version 6.1. Chicago Ill: SPSS Inc, 1993.
  23. Campbell MJ, Gardner MJ. Calculating confidence intervals for some non-parametric analyses. BMJ 1988; 296: 1454-1456.
  24. Calman KC. Cancer: science and society and the communication of risk. BMJ 1996; 313: 799-802.
  25. Ayanian JZ, Landrum MB, Normand ST, et al. Rating the appropriateness of coronary angiography -- do practicing physicians agree with an expert panel and with each other? N Engl J Med 1998; 338: 1896-1904.
  26. Meisel A, Kuczewski M. Legal and ethical myths about informed consent. Arch Intern Med 1996; 156: 2521-2526.
  27. Shahar E. A Popperian perspective of the term 'evidence-based medicine'. J Eval Clin Pract 1997; 2: 109-116.

(Received 1 Mar, accepted 19 Aug, 1999)

Authors' Details North Western Adelaide Health Service, Queen Elizabeth Hospital, Adelaide, SA.
Stephen P Fitzgerald, MB BS, FRACP, Visiting Medical Specialist, Endocrinology;
George Phillipov, MSc, PhD, Chief Medical Scientist, Endocrinology.

Reprints: Dr G Phillipov, Endocrinology, The Queen Elizabeth Hospital, Woodville, SA 5011.
gphillipovATtqehsmtp.tqeh.sa.gov.au






1: Demographics of respondents in the three surveys
Survey 1
Survey 2
Survey 3
Interventions surveyed
HRT
HRT
HRT
THROM
THROM
CABS
CABS
Number asked to participate
100
82
98
Refusals*
15
5
3
Actual participants
85
77
95
Setting
Hospital outpatients clinics
April 1997
General practice surgeries
May 1997
General practice surgery
October 1997
Median age in years (range)
60 (17-88)
47 (17-78)
39 (17-77)
Number of men
34 (41%)
23 (31%)
40 (42%)
Number born in Australia
55 (68%)
57 (78%)
76 (80%)
Number reporting heart disease
14 (17%)
9 (12%)
6 (6%)
Number reporting heart surgery
11 (13%)
6 (8%)
2 (2%)


*As reported by nursing and reception staff. HRT=hormone replacement therapy. THROM=thrombolysis for acute myocardial infarction. CABS=coronary artery bypass surgery.
Back to text

2: Hormone replacement therapy: survey questions and main results

A: Surveys 1 and 2

Level of evidence:2 III-2.13-15
There was no difference between groups in surveys 1 and 2 in:

  • age (Mann-Whitney U test, P=0.70);
  • sex (Fisher's exact test, P=0.40);
  • being Australian-born (Fisher's exact test, P=0.44).
The difference in median scores was -0.5 (95% CI, -2.1 to +1.0), so data from the two surveys were pooled.
Question: Imagine you are age 50. You are offered a therapy which over the next ten years should reduce your risk of heart disease from 10% to 5%. Unfortunately, the safety of this treatment (tablets which you must take every day) is not proven. There is a controversial belief that these tablets might cause cancer in about 2% of individuals. In summary, the tablets have a 5% chance of being helpful, a possible 2% chance of doing harm and a 93% chance of doing neither harm nor good.* How do you think you would feel about such treatment if it were offered to you?
Analysis exclusions: 104 patients with age >50 years or reported heart disease.
Participants: 58 patients; median age, 38.5 years (range, 17-49 years); 20 (35%) men; 47 (86%) born in Australia.
Individual scores independent of:
  • sex (Mann-Whitney U test, P=0.41);
  • age (Kendall correlation coefficient, -0.12; P=0.24);
  • being Australian-born (Mann-Whitney U test, P=0.87).
Median response:‡ -2.95 (95% CI, -5 to -2.1).

Free-text responses: 32 (55%) responses.
Main free-text themes:† 17 (53%), aversion to taking any risk; 7 (22%), prefer alternative medicine procedures.
  • Patients who indicated that their attitude was influenced by the risk of cancer as a side-effect, and those who indicated support for alternative medicine approaches, had a median score of -5.

B: Survey 3

Question: Imagine you are age 50. You are offered a therapy which over the next 15 years should reduce your risk of heart disease from 20% to 10%. Unfortunately the safety of this treatment (tablets which you must take every day) is not proven. There is a controversial belief that these tablets might cause cancer in about 1% of individuals. In summary, the tablets have a 10% chance of being helpful, a possible 1% chance of doing harm and an 89% chance of doing neither harm nor good. How do you think you would feel about such treatment if it were offered to you?
Analysis exclusions: 27 patients with age >50 years or reported heart disease.
Participants: 68 patients; median age, 36 years (range, 17-49 years); 26 (38%) men; 56 (82%) born in Australia.
Median response:‡ -0.75 (95% CI, -2.3 to 0).

  • Median score significantly higher than for surveys 1 and 2 (Mann-Whitney U test, P<0.01)


*"No effect" statement removed for survey 2. †See Methods for details of the theme categories. ‡Responses ranged from -5 (definitely would not accept the treatment) to +5 (definitely would accept the treatment).
Back to text

3: Thrombolysis for acute myocardial infarction, and coronary artery bypass surgery: survey questions and main results

Responses for surveys 1 and 2 were combined.
Analysis exclusions: 51 patients with age >65 years.
Participants: 111 patients; median age, 47 years (range, 17-64 years); 37 (34%) men; 79 (75%) born in Australia; 21 (19%) reported cardiovascular disease.
Individual scores independent of:

  • sex (Mann-Whitney U test, P=0.07);
  • age (Kendall correlation coefficient, P=0.19);
  • being Australian-born (Mann-Whitney U test, P=0.70);
  • prior heart surgery (Mann-Whitney U test, P=0.61).

Thrombolysis for acute myocardial infarction
Level of evidence:2 II.16
Question: Imagine you are 65. You are brought to hospital with severe chest pain and it is thought that you are having a heart attack. You are offered an injection which should reduce your chance of dying from 10% to 7%, but unfortunately has a side-effect of causing serious bleeding in about 1% of patients. If, for example, this bleeding occurs in the brain, it might cause a stroke or death. In summary there is a 3% chance of the injection saving your life, a 1% chance of causing serious bleeding and a 96% chance that it will do neither*. How would you feel about such an injection?
Median response:‡ -0.5 (95% CI, -0.9 to 0)


Free-text response: 55 (50%) responses.
Main free-text themes:† 14 (25%), odds of success given; 12 (22%), aversion to taking any risk; 11 (20%) willing to take any risk for health.

Coronary artery bypass surgery
Level of evidence:2 II.17
Question: Imagine in the above scenario that you survived the heart attack which fortunately was small and that you now feel perfectly well. As part of follow-up tests it is discovered that you have blockages in your coronary arteries (blood vessels to the heart). It is possible to unblock these arteries with major surgery. This surgery has about 1 in 10 chance of saving your life sometime over the next 5 years. Unfortunately as well as having to endure major surgery, there is a 1 in 20 chance that the surgery will cause death or a stroke. In summary there is a 10% chance that the surgery will be helpful, a 5% chance that it will cause harm and an 85% chance that it will make no difference*. How would you feel about an operation?
Median response:‡ -0.1 (95% CI, -0.5 to +1.3)


Free-text response: 46 (41%) responses.
Main free-text themes:† 10 (22%), odds of success given; 9 (20%), aversion to taking any risk.
  • Patient attitude to THROM and CABS was less negative than for HRT (Wilcoxon matched-pairs signed-ranks test, P<0.001).
  • Difference in median scores between patients with and without heart disease was significantly increased for CABS (+2.0; 95% CI, 0 to +3.8), but not for THROM (+1.2; 95% CI, -0.6 to +2.8).


*"No effect" statement removed for survey 2. †See Methods for details of the theme categories. ‡Responses ranged from -5 (definitely would not accept the treatment) to +5 (definitely would accept the treatment). CABS=coronary artery bypass surgery. HRT=hormone replacement therapy. THROM=thrombolysis for acute myocardial infarction.
Back to text


Authors


References

  1. Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 1992; 268: 2420-2425.
  2. National Health and Medical Research Council. A guide to the development, implementation and evaluation of clinical practice guidelines. Canberra: Commonwealth of Australia, 1999. Available at <http://www.nhmrc.health.gov.au/publicat/synopses/cp30syn.htm>.
  3. Bogardus Jr ST, Holmboe E, Jekel JF. Perils, pitfalls, and possibilities in talking about medical risk. JAMA 1999; 281: 1037-1041.
  4. McNeil BJ, Weichselbaum R, Pauker SG. Fallacy of the five-year survival in lung cancer. N Engl J Med 1978; 299: 1397-1401.
  5. McNeil BJ, Weichselbaum R, Pauker SG. Speech and survival: tradeoffs between quality and quantity of life in laryngeal cancer. N Engl J Med 1981; 305: 982-987.
  6. Schwartz LM, Woloshin S, Black WC, Welch HG. The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med 1997; 127: 966-972.
  7. Searle J. Routine antenatal screening: not a case of informed choice. Aust N Z J Pub Health 1997; 21: 268-274.
  8. Marteau TM, Kidd J, Cook R, et al. Perceived risk not actual risk predicts uptake of amniocentesis. Br J Obstet Gynaecol 1991; 98: 282-286.
  9. Slaytor EK, Ward JE. How risks of breast cancer and benefits of screening are communicated to women: analysis of 58 pamphlets. BMJ 1998; 317: 263-264.
  10. Col NF, Eckman MH, Wong JB, Pauker SG. Strategies for individualizing patient decisions about hormone therapy. J Clin Endocrinol Metab 1999; 84: 1799-1802.
  11. Antman EM, Braunwald E. Acute myocardial infarction. In: Fauci AS, Braunwald E, Isselbacher KJ, et al, editors. Harrison's principles of internal medicine. 14th ed. New York: McGraw-Hill, 1998; 1352-1365.
  12. Loop FD. The surgical treatment of artherosclerotic coronary heart disease. In: Schlant RC, Alexander RW, editors. Hurst's The heart. 8th ed. New York: McGraw-Hill, 1994; 1367-1380.
  13. Grodstein F, Stampfer MJ, Manson JE, et al. Postmenopausal estrogen and progestin use and the risk of cardiovascular disease. N Engl J Med 1996; 335: 453-461.
  14. Falkeborn M, Persson I, Adami H-O, et al. The risk of acute myocardial infarction after oestrogen and oestrogen-progestogen replacement. Br J Obstet Gynaecol 1992; 99: 821-828.
  15. Colditz GA, Markinson SE, Hunter DJ, Willett WC. The use of oestrogens and progestins and the risk of breast cancer in post-menopausal women. N Engl J Med 1995; 332: 1589-1593.
  16. Gruppo Italiano per lo Studio della Streptochinasi nell'Infarto Miocardico (GISSI). Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet 1986; i: 397-402.
  17. The Veterans Administration Coronary Artery Bypass Surgery Cooperative Study Group. Eleven-year survival in the Veterans Administration randomised trial of coronary bypass surgery for stable angina. N Engl J Med 1984; 311: 1333-1339.
  18. Forrow L, Taylor WC, Arnold RM. Absolute relative: how research results are summarized can affect treatment decisions. Am J Med 1992; 92: 121-123.
  19. Malenka DJ, Baron JA, Johansen S, et al. The framing effect of relative and absolute risk. J Gen Intern Med 1993; 8: 543-548.
  20. Skolbekken J-A. Communicating the risk reduction achieved by cholesterol reducing drugs. BMJ 1998; 316: 1956-1958.
  21. Bunker JP, Houghton J, Baum M. Putting the risk of breast cancer in perspective. BMJ 1998; 317: 1307-1309.
  22. SPSS [computer program]. Version 6.1. Chicago Ill: SPSS Inc, 1993.
  23. Campbell MJ, Gardner MJ. Calculating confidence intervals for some non-parametric analyses. BMJ 1988; 296: 1454-1456.
  24. Calman KC. Cancer: science and society and the communication of risk. BMJ 1996; 313: 799-802.
  25. Ayanian JZ, Landrum MB, Normand ST, et al. Rating the appropriateness of coronary angiography -- do practicing physicians agree with an expert panel and with each other? N Engl J Med 1998; 338: 1896-1904.
  26. Meisel A, Kuczewski M. Legal and ethical myths about informed consent. Arch Intern Med 1996; 156: 2521-2526.
  27. Shahar E. A Popperian perspective of the term 'evidence-based medicine'. J Eval Clin Pract 1997; 2: 109-116.