Drug interactions can be dangerous and at times lethal.1,2 Drug interaction alert messages in clinical software may help clinicians decide whether certain drug combinations can be used safely. Most clinical software systems include drug interaction decision support; however, they use different reference sources and they implement and present drug interaction information in different ways.3,4 Some clinicians have said that they are bombarded with too many alerts that they find annoying and that the information provided is sometimes irrelevant or unhelpful.5-7 Despite this, general practitioners and pharmacists generally believe the benefits are sufficiently important for them not to switch off alerts.8,9
There are currently no standards or guidelines in Australia pertaining to the quality or suitability of drug interaction knowledge bases for decision support, and little guidance on how such knowledge bases should be implemented in clinical software (eg, how or when information is displayed to users). In addition, a recent study found that drug interaction decision support in commonly used prescribing and dispensing systems in Australia has significant shortcomings; namely, variation between systems, low specificity of alerts in some systems, and a lack of information on clinical effects and management advice.4 An expert panel made recommendations for improving the content and format of drug interaction alerts.4
We aimed to find out whether Australian GPs and pharmacists agreed with these recommendations and to explore their preferences in relation to the content, format and usability of drug interaction alerts. While there are numerous barriers and facilitators to the uptake of decision support,10 we were particularly interested in the format of drug interaction alerts because format can influence how users respond to and use decision support11,12 and research in this area is limited. We also aimed to develop recommendations for software vendors and knowledge providers that would lead to more useful and acceptable decision support for GPs and pharmacists.
A postal survey for GPs and community pharmacists was developed, with the content informed by expert panel recommendations4 and a literature review (conducted in February 2010) on drug interaction decision support. The key survey questions are shown in Box 1; demographic information was also collected. The survey questions were the same for GPs and pharmacists, apart from the eligibility questions (which were worded differently to reflect prescribing and dispensing processes). The survey was piloted for face validity by two GPs and two community pharmacists.
GPs and pharmacists were eligible to participate in the study if they currently practised, used a computer to prescribe or perform the data entry step of dispensing, and had drug interaction alerts switched on in their prescribing or dispensing software. A standard sample-size calculator for prevalence surveys13 was used to generate required sample-sizes of 244 GPs and 242 pharmacists, with four assumptions: 80% of respondents would agree that four key components of information in a drug interaction alert4 are useful; Australian GP and community pharmacist populations of 24 000 and 13 000, respectively; a 95% confidence interval; and 5% precision. We multiplied the required sample sizes by four, based on an expected response rate of 25%, and rounded these numbers up to 1000. The postal survey was sent to 1000 GPs and 1000 pharmacists who had been randomly selected from a comprehensive database of health professionals practising in Australia (AMPCo Direct).
Surveys were returned by 219 GPs and 170 pharmacists (22% and 17% response rates, respectively). After excluding surveys from ineligible respondents, 191 GP surveys and 138 pharmacist surveys were analysed. Characteristics of respondents included in the analysis are shown in Box 2. Cross-checking of machine-scanned data showed a 0.17% error rate.
GPs and pharmacists believed that the most useful components of information in drug interaction alerts were severity, clinical effects and management advice (Box 3). Also useful — but not to the same extent — were information on timeframe, mechanism, the date of the last update of the information source and access to a list of reference sources. Some GPs and pharmacists indicated that information on the frequency or likelihood of adverse effects arising from drug interactions would also be useful.
Most GPs and pharmacists preferred drug interaction alerts to be in the “headings + bullets” format (Box 4). Reasons provided were that this format is clear, concise, easy to scan through or read, and easy to navigate. Some respondents indicated that they wanted the alerts in their systems to better draw their attention — for example, larger pop-up boxes and text size, and more colour and visual effects. Some also wanted changes that would reduce the need for scrolling and changes in the amount of information the drug interaction alert displayed initially, to aid more rapid comprehension of information.
The levels of agreement among respondents on the value of various usability features in drug interaction decision support are shown in Box 5. Several GPs stated that when there are multiple drug interaction alerts for a patient, those with the most severe warning should appear first. Some respondents suggested ways to customise drug interaction alerts, including suppression of minor and moderate alerts, alerts on repeat prescriptions and alerts that have appeared more than once or twice for a patient.
In this study, GPs and pharmacists indicated what is important to them in relation to drug interaction decision support in clinical software, including components of information they would find useful, the best way to present information, and usability features that they would value. They concurred with previous recommendations4 — in particular, almost all GPs and pharmacists wanted information on clinical effects and management advice, which is not consistently available in commonly used general practice and pharmacy software systems.
GPs and pharmacists also wanted information on severity, including differentiation of drug interaction alerts by severity. Information on the severity of an interaction has been used to prioritise and increase the acceptance of alerts.14 However, there has been no comprehensive assessment of the validity of severity ratings15 and studies have shown a high degree of discordance between different reference sources in the severity ratings assigned to drug interactions.15,16 There are a number of reasons for this, including a lack of good epidemiological evidence for drug interactions and the use of variable term-inology and classification systems.15,17 Given the overwhelming support from GPs and pharmacists for severity information to be available, this should be explored further and efforts should be made to ensure that severity ratings are consistent and reliable. In addition, current users of drug interaction alerts should be made aware of the limitations of severity ratings.
GPs and pharmacists perceived some drug interaction alerts to be unhelpful or irrelevant to decision making. Reasons for this included irrelevant content and alerts for clinically unimportant interactions. A lack of sophistication in decision support systems can also be a factor; for example, systems may not recognise drugs that a patient is no longer taking or that have already been prescribed in another strength. While there are valid reasons for intentionally overriding alerts,18 clinicians may also ignore alerts because of alert fatigue.19,20 This can have serious implications if critical drug interaction alerts are inadvertently bypassed.
More than 80% of GPs and pharmacists in our study thought it should be more difficult for users to override severe interaction alerts. One way to do so is to make it mandatory for users to provide a reason when they override a severe interaction alert. Users may find it useful to record their actions as a way of reminding themselves or informing others of their decision and to justify their reason for overriding an alert. About half the respondents agreed or strongly agreed that users should be able to customise drug interaction alerts (eg, suppress an alert if the user is already familiar with it). Some users may welcome the option of tailoring which drug interactions generate alerts, while the fear of missing potentially important interactions may deter others.5,9
Our findings support those of others. It has previously been shown that clinicians want information on management advice and severity of interactions.21-23 Many clinicians would like to be able to differentiate drug interaction alerts by severity, increase the difficulty of overriding alerts for severe interactions, and make it mandatory for users to provide a reason for doing so.7,9,23,24 Clinicians’ frustration with irrelevant or unhelpful alerts has also been reported.6,7,25
Our findings on the presentation of alert messages aligns with recent research showing that the way information is displayed to users affects workflow and acceptance of decision support.11,12 In an observational study of users’ responses to medication alerts, it was found that a key barrier to workflow and decision making was poor screen display.11 Another study showed a strong correlation between the quality of the display and whether an alert was accepted or not.12 Our study highlighted that changes to the format of an alert message can improve readability and navigation without altering the content.
This study offers an Australian perspective on the issues of content, format and usability of drug interaction decision support and provides evidence of shortcomings that drug interaction knowledge providers and software vendors need to address for users. Drug interaction alerts could be improved by changes to content and format according to our recommendations (Box 7).
1 Key survey questions on content, format and usability of drug interaction decision support
Severity of the interaction
Access to a list of reference sources
Date that information was last updated
Others (please specify)
Ability to customise drug interaction alerts
Links to more detailed information
Ability to differentiate drug interaction alerts from other types of warning messages
Ability to differentiate drug interaction alerts by severity
Make it harder to override alerts for severe interactions
Make it mandatory for users to provide a reason when overriding an alert for a severe interaction
NPS = National Prescribing Service. * Examples of the three formats are shown in Box 4.
3 Proportions of general practitioners and pharmacists who considered various components of drug interaction alerts to be mostly or sometimes useful*
 | |||||||||||||||
|
* Missing data and data for the responses “not useful” and “not sure” are not shown. | |||||||||||||||
4 General practitioners’ and pharmacists’ preferences for drug interaction alert formats*
Example: methotrexate and trimethoprim |
|||||||||||||||
5 Levels of agreement among general practitioners and pharmacists on the value of various drug interaction alert usability features*
 | |||||||||||||||
|
* Missing data and data for the response “not sure” are not shown. | |||||||||||||||
6 General practitioners’ and pharmacists’ comments on various aspects of drug interaction decision support
“Keep it brief and to the point. Don’t have time to read large amounts of information.” (Pharmacist)
“[Drug] interaction alerts should be the same for all software.” (Pharmacist)
7 Recommendations for software vendors and knowledge providers to improve drug interaction decision support, and areas for further investigation
Content: Drug interaction messages should include information on clinical effects, management advice, timeframe and mechanism, where this information is known. Inclusion of information on severity requires further investigation.
Format: Information should be presented in an easy to read format that includes headings (eg, “clinical effects”, “management advice”) and bullet points.
Relevance of information: Drug interaction knowledge bases should include only clinically significant drug interactions. They should be structured so that warnings can be provided for individual drugs (where not all drugs in a class interact in the same way) and for the prescribed route of administration (eg, oral formulations but not creams and ointments, where applicable).
Reduction of alert fatigue: There should not be warnings for drugs that are not currently used by the patient, nor duplicate warnings where one drug is prescribed in different strengths (eg, warfarin).
Areas for further investigation
Further work is required to ensure that severity ratings are reliable for users. Health professionals should be aware that severity is context-dependent and that severity ratings can vary markedly between different prescribing and dispensing systems; caution is needed when interpreting information on severity.
Customisation of drug interaction decision support by users may be useful but more research on how this can be done safely in practice is needed.
Participants in this study believed that it should be made harder for users to override severe drug interaction alerts and that it should be mandatory for users to provide a reason when they do so. How this is best implemented requires further investigation.
Received 24 February 2011, accepted 6 October 2011
- Kitty H Yu1
- Michelle Sweidan1
- Margaret Williamson2
- Amanda Fraser3
- 1 e-Health and Decision Support, National Prescribing Service, Melbourne, VIC.
- 2 Research and Development, National Prescribing Service, Sydney, NSW.
- 3 South Yarra Medical, Melbourne, VIC.
We thank James Reeve, Jonathan Dartnell and Malcolm Gillies (National Prescribing Service) for helpful comments in preparation of this manuscript.
No relevant disclosures.
- 1. New South Wales Department of Health. Safety notice: 011/09. Allopurinol and azathioprine. 7 May 2009. http://www.health.nsw.gov.au/resources/quality/sabs/pdf/sn20090507.pdf (accessed Nov 2011).
- 2. Pilgrim JP, Gerostamoulos D, Drummer OH. Deaths involving contraindicated and inappropriate combinations of serotonergic drugs. Int J Legal Med 2011; 125: 803-815.
- 3. Saverno KR, Hines LE, Warholak TL, et al. Ability of pharmacy clinical decision-support software to alert users about clinically important drug-drug interactions. J Am Med Inform Assoc 2010; 18: 32-37.
- 4. Sweidan M, Reeve JF, Brien JE, et al. Quality of drug interaction alerts in prescribing and dispensing software. Med J Aust 2009; 190: 251-254. <MJA full text>
- 5. Ahearn MD, Kerr SJ. General practitioners’ perceptions of the pharmaceutical decision-support tools in their prescribing software. Med J Aust 2003; 179: 34-37. <MJA full text>
- 6. Lapane KL, Waring ME, Schneider KL, et al. A mixed method study of the merits of e-prescribing drug alerts in primary care. J Gen Intern Med 2008; 23: 442-446.
- 7. Magnus D, Rodgers S, Avery AJ. GPs’ views on computerized drug interaction alerts: questionnaire survey. J Clin Pharm Ther 2002; 27: 377-382.
- 8. Glassman PA, Belperio P, Simon B, et al. Exposure to automated drug alerts over time: effects on clinicians’ knowledge and perceptions. Med Care 2006; 44: 250-256.
- 9. Weingart SN, Massagli M, Cyrulik A, et al. Assessing the value of electronic prescribing in ambulatory care: a focus group study. Int J Med Inform 2009; 78: 571-578.
- 10. Moxey A, Robertson J, Newby D, et al. Computerized clinical decision support for prescribing: provision does not guarantee uptake. J Am Med Inform Assoc 2010; 17: 25-33.
- 11. Russ AL, Zillich AJ, McManus MS, et al. A human factors investigation of medication alerts: barriers to prescriber decision-making and clinical workflow. AMIA Annu Symp Proc 2009; 2009: 548-552.
- 12. Seidling HM, Phansalkar S, Seger DL, et al. Factors influencing alert acceptance: a novel approach for predicting the success of clinical decision support. J Am Med Inform Assoc 2011; 18: 479-484.
- 13. Glaziou P. Sample size for a prevalence survey, with finite population correction. http://sampsize.sourceforge.net/iface/index.html (accessed Mar 2010).
- 14. Paterno MD, Maviglia SM, Gorman PN, et al. Tiering drug–drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009; 16: 40-46.
- 15. Vitry AI. Comparative assessment of four drug interaction compendia. Br J Clin Pharmacol 2007; 63: 709-714.
- 16. Wang LM, Wong M, Lightwood JM, et al. Black box warning contraindicated comedications: concordance among three major drug interaction screening programs. Ann Pharmacother 2010; 44: 28-34.
- 17. Abarca J, Malone DC, Armstrong EP, et al. Concordance of severity ratings provided in four drug interaction compendia. J Am Pharm Assoc (2003) 2004; 44: 136-141.
- 18. Weingart SN, Toth M, Sands DZ, et al. Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med 2003; 163: 2625-2631.
- 19. Weingart SN, Simchowitz B, Shiman L, et al. Clinicians’ assessments of electronic medication safety alerts in ambulatory care. Arch Intern Med 2009; 169: 1627-1632.
- 20. Cash JJ. Alert fatigue. Am J Health Syst Pharm 2009; 66: 2098-2101.
- 21. Bergk V, Gasse C, Schnell R, et al. Requirements for a successful implementation of drug interaction information systems in general practice: results of a questionnaire survey in Germany. Eur J Clin Pharmacol 2004; 60: 595-602.
- 22. Indermitte J, Erba L, Beutler M, et al. Management of potential drug interactions in community pharmacies: a questionnaire-based survey in Switzerland. Eur J Clin Pharmacol 2007; 63: 297-305.
- 23. Ko Y, Abarca J, Malone DC, et al. Practitioners’ views on computerized drug–drug interaction alerts in the VA system. J Am Med Inform Assoc 2007; 14: 56-64.
- 24. Murphy JE, Forrey RA, Desiraju U. Community pharmacists’ responses to drug–drug interaction alerts. Am J Health Syst Pharm 2004; 61: 1484-1487.
- 25. Abarca J, Malone DC, Skrepnek GH, et al. Community pharmacy managers’ perception of computerized drug–drug interaction alerts. J Am Pharm Assoc (2003) 2006; 46: 148-153.
Abstract
Objective: To explore Australian general practitioners’ and pharmacists’ preferences in relation to content, format and usability of drug interaction alerts in prescribing and dispensing software.
Design, participants and setting: Surveys that sought opinions on drug interaction decision support were mailed to a random sample of GPs and community pharmacists (1000 of each) in June 2010.
Main outcome measures: Usefulness of various components of drug interaction information; preferred format of drug interaction alerts; levels of agreement on the value of various usability features; aspects of drug interaction decision support users would most like to change.
Results: Surveys were returned by 219 GPs and 170 pharmacists. Of the 191 GPs and 138 pharmacists included in the analysis, the vast majority considered severity, clinical effects and management advice to be mostly or sometimes useful in drug interaction alerts. The most popular drug interaction alert format — favoured by 131 GPs (69%) and 115 pharmacists (83%) — was one with headings and one or two succinct bullet points under each. The vast majority of respondents also wanted to be able to differentiate drug interaction alerts by severity, and a majority agreed that it should be made more difficult to override alerts for severe interactions and that it should be mandatory to provide a reason for doing so.
Conclusions: GPs and pharmacists want drug interaction alert information to be relevant, useful, concise, and easy to read and comprehend. Software vendors and knowledge providers could improve drug interaction decision support by making changes to the content and format of drug interaction alerts according to our recommendations.