Objective: To investigate the quality of drug interaction decision support in selected prescribing and dispensing software systems, and to compare this information with that found in a range of reference sources.
Design and setting: A comparative study, conducted between June 2006 and February 2007, of the support provided for making decisions about 20 major and 20 minor drug interactions in six prescribing and three dispensing software systems used in primary care in Australia. Five electronic reference sources were evaluated for comparison.
Main outcome measures: Sensitivity, specificity and quality of information; for major interactions: whether information on clinical effects, timeframe and pharmacological mechanism was included, whether management advice was helpful, and succinctness.
Results: Six of the nine software systems had a sensitivity rate ≥ 90%, detecting most of the major interactions. Only 3/9 systems had a specificity rate of ≥ 80%, with other systems providing inappropriate or unhelpful alerts for many minor interactions. Only 2/9 systems provided adequate information about clinical effects for more than half the major drug interactions, and 1/9 provided useful management advice for more than half of these. The reference sources had high sensitivity and in general provided more comprehensive clinical information than the software systems.
Drug interaction alerts are a basic form of electronic decision support in clinical software, usually in the form of a “pop-up” message that appears when interacting drugs are prescribed or dispensed. Drug interaction decision support has considerable potential for improving patient safety, but there is concern about variation between systems1 and about the quality and usefulness of the information. General practitioners and pharmacists in Australia have complained about irrelevant drug interaction alerts, and lack confidence in the comprehensiveness and accuracy of the information.2,3 Studies have identified problems relating to the quality, relevance and useability of information in interaction alerts4-7 and inconsistencies between references on drug interactions.8,9
To be useful, a drug interaction decision support system must be both sensitive (to alert the user to potential clinically significant interactions) and specific (to avoid inundation with irrelevant alerts). In Australia, there are no standards for clinical software10 or for the integration of decision support tools into software,11 nor is there a national drug interaction database that software vendors can use (as there is in the Netherlands12).
We investigated the drug interaction decision support in prescribing and dispensing software used in primary care in Australia, to examine the quality and usefulness of the information, to compare the information in alerts with that found in a range of reference sources, and to investigate variations between systems, including sensitivity and specificity.
The study was conducted between June 2006 and February 2007. The researchers (M S and J F R) worked with an expert panel comprising a clinical–academic pharmacist (J E B), a GP (P J), a clinical pharmacologist–physician (J H M) and a drug information pharmacist (G M V). The panel’s role was to comment on the study design, assess the data, and make recommendations.
Six prescribing systems — Best Practice (Best Practice Software, Bundaberg, Qld), Genie (Genie Solutions, Brisbane, Qld), Medical Director 2 (Health Communication Network, Sydney, NSW), MedTech32 (Medtech Global, Melbourne, Vic), Plexus (IBA Health, Sydney, NSW) and Profile (Intrahealth Systems, Sydney, NSW) — and three dispensing systems — Amfac Dispense (Corum Health Services, Sydney, NSW), Pharmasol LOTS (Corum Health Services, Sydney, NSW) and WiniFRED (PCA NU Systems, Melbourne, Vic) — were selected. It was estimated that 80%–90% of GPs3,13 and 65% of community pharmacists (Mike Farrell, Councillor, Pharmacy Guild of Australia, personal communication) in Australia used one of these systems.
Five electronic reference sources were selected for comparison — three Australian publications commonly used by doctors and pharmacists, and two publications from the United Kingdom. The Australian Medicines Handbook14 contains brief information on drug interactions, which is intended to be used in conjunction with the full drug monographs in the handbook. MIMS DrugAlert Interactions15 is a specialist knowledgebase on drug interactions that is integrated into some software systems. Product information15 is the official source of drug information in Australia, and is approved by the Therapeutic Goods Administration. Stockley’s Drug Interactions,16 published in the UK, is a comprehensive source widely used by drug information pharmacists, with a companion clinical decision support tool, Stockley’s Interaction Alerts.17
Twenty “major” (defined here as clinically significant) and 20 “minor” (clinically unimportant) drug interactions were assessed in each software system (Box 1). The selection criteria for major interactions were: both drugs used in primary care + pair is potentially used in combination + at least one drug in pair is commonly used or is a new drug or the pair must produce a potentially serious adverse outcome + interaction is classified as “moderate” or “severe” in Stockley’s Interaction Alerts17 and “moderate” or “major” in Drug Interaction Facts.18 Selection criteria for minor interactions were: both drugs are used in primary care + pair is potentially used in combination + at least one drug in pair is commonly used or is a new drug + interaction is classified as “minor” in Drug Interaction Facts18 OR one drug is a topical product and topical use is unlikely to result in a clinically significant interaction.
The major interactions represent a range encountered in practice, not necessarily those that are most common or most severe. Potential drug pairs (47 major and 51 minor interactions) were identified on the basis of our clinical experience, reports to the Adverse Drug Reactions Advisory Committee, similar published studies, and whether the drugs are commonly prescribed.19 A modified Delphi process with the expert panel members was used to remove drug pairs to produce the 40 interactions for testing.
Testing was conducted between September and November 2006 with the latest version of each software system. If the system allowed customisation of the severity level of drug interactions to be detected, the minimum setting was selected (to detect the maximum number of interactions).
A new “dummy” patient was entered into each system. Each of the drug pairs was prescribed or dispensed for this patient, and deleted before the next pair was entered. The contents of any drug interaction alerts were copied to a spreadsheet. Any further information that could be obtained by clicking on a link was not assessed, so as to compare like with like, and because it was not known how often users click on links.
Each reference source was searched for information about each drug pair, and any information found was copied to a spreadsheet. If an individual drug was not found, the drug class was checked. As for software systems, if the reference source allowed customisation of the severity level of drug interactions to be shown, the minimum setting was selected. The interaction information from the product information of the first drug in each pair was assessed.
Sensitivity and specificity were calculated similarly to other studies investigating drug interaction software.20,21 Sensitivity was defined as the proportion of the 20 major interactions detected (ie, triggering an alert in the software system or noted in the reference source). Specificity was defined as the proportion of the 20 minor interactions not detected (ie, not triggering an alert in the software or noted in the reference source), or detected but including an explanation of why the interaction is not important in clinical practice.
For major drug interactions, the expert panel assessed whether information was included (yes, no, not sure) about clinical effects, timeframe (onset or duration of effects), and pharmacological mechanism; and whether management advice was helpful (useful, somewhat useful, unhelpful, none, incorrect). The panel considered that these aspects constituted key information for clinical decision making and were likely to be known for many or all of the drug interactions. Succinctness of the information was also assessed (yes, no, not sure).
Panel members were blinded to the identity of the software systems. Each drug interaction was rated for the five parameters; the information was considered from a practitioner’s perspective in a practice setting. The group met after assessing the information individually. Where their ratings differed, consensus was reached by discussion. Summary statistics of the results were calculated with SPSS, version 15.0 (SPSS Inc, Chicago, Ill, USA).
Five of the six prescribing systems had high sensitivity, detecting ≥ 90% of the major drug interactions (Box 2). The systems were more variable and generally performed less well with minor interactions, with specificity ranging from 25% to 85%. For example, in the system with a specificity of 25%, alerts came up for 75% (15/20) of the minor interactions, and the information provided did not indicate that the interaction was not important in clinical practice.
Only one of the three dispensing systems detected ≥ 90% of the major interactions (Box 2). In general, their specificity was better than the prescribing systems (range, 65%–85%).
Only two of the nine software systems provided adequate information about clinical effects for more than half of the major drug interactions (Box 3). Little or no information about the timeframe of interactions was provided, even in a comprehensive reference source such as Stockley’s Drug Interactions, which had this information for 11/20 interactions. Only one software system described the pharmacological mechanism for most interactions (15/20). Overall, the software systems did not provide useful management advice, most providing advice considered “somewhat useful” or none at all. Four software systems provided links to additional information: two to the product information, and two to more detailed information from the drug interactions knowledgebase used. This information was not assessed.
In some cases, the language used in alerts lacked clarity and succinctness, and was occasionally incomprehensible. Some systems used unhelpful phrases such as “use with caution”, spelled drug names incorrectly, or named drugs that are not available in Australia.
In general, the reference sources performed better than the software systems in terms of the comprehensiveness of information provided (Box 3).
Most of the prescribing and dispensing software systems assessed did not offer consistently useful, relevant support to GPs and pharmacists for making decisions about potentially interacting drugs. The systems varied markedly, due in part to the use of different knowledgebases but also to variable use of the same knowledgebase. Four of the prescribing systems used MIMS DrugAlert Interactions, but different implementations produced markedly different results.
It is cause for concern that three of the nine systems detected < 90% of major interactions. The relatively low specificity rates (< 80% for six systems) suggest that some software systems contain many inappropriate or unhelpful alerts. Examples are inappropriate alerts for interactions with topical products (eg, terbinafine cream + warfarin), for minor pharmacokinetic interactions (eg, warfarin + spironolactone), and failure to differentiate between drugs in a class (eg, digoxin + felodipine versus other calcium channel blockers).
This study had some limitations. The panel members were selected for their expert knowledge and clinical experience, but they are not necessarily representative of software users. The rating of information was subjective, although a consistent approach was used and consensus was achieved.
Our results support international literature in this area. Studies have shown variation and deficiencies in drug interaction decision support in pharmacy software systems in the United States,22 in clinical software systems used in primary care in Canada,23 and in drug interaction software for personal digital assistants.20,21 A study of German GPs found that they wanted detailed drug interaction information, but their needs were not being met for some components (eg, management of interactions, outcomes).24
The expert panel has made recommendations to publishers of drug interaction information and software vendors to improve drug interaction decision support (Box 4). Most software vendors rely on the drug interaction knowledgebases they purchase, so publishers must ensure provision of accurate, useful, up-to-date information, and vendors should implement the information appropriately. Standards or guidelines for decision support in software could improve quality and result in greater consistency between software systems.
Deficiencies in drug interaction decision support can impede the quality use of medicines, in terms of individual patient management and by causing alert fatigue, desensitising users to prompts and alerts. It is important that doctors and pharmacists are aware of the limitations of the decision support features in their software systems, including the quality and source of the underlying evidence, and refer to alternative sources of information if required.
1 Drug interactions assessed
Major drug interactions: azathioprine + allopurinol; colchicine + clarithromycin; cyclosporin + diltiazem; digoxin + clarithromycin; eplerenone + ketoconazole; ethinyloestradiol/ levonorgestrel* + doxycycline; fluoxetine + sibutramine; indinavir + St John’s wort; lithium + diclofenac; methadone + phenytoin; methotrexate + trimethoprim; sertraline + tramadol; sildenafil + isosorbide mononitrate; simvastatin + gemfibrozil; spironolactone + perindopril; theophylline + erythromycin; verapamil + metoprolol; warfarin + amiodarone; warfarin + fluconazole; warfarin + thyroxine
Minor drug interactions: amoxycillin + erythromycin; aspirin (low dose) + ethinyloestradiol/ levonorgestrel*; aspirin (low dose) + phenytoin; aspirin (low dose) + spironolactone; erythromycin skin lotion + amitriptyline; felodipine + digoxin; felodipine + metoprolol; frusemide + ciprofloxacin; hydrochlorothiazide* (with irbesartan) + doxycycline; metformin + oxybutynin; omeprazole + nifedipine; paracetamol + oxybutynin; paroxetine + oxazepam; prednisolone + ethinyloestradiol/levonorgestrel*; ranitidine + ferrous sulfate; ranitidine + metoprolol; simvastatin + glibenclamide; temazepam + levodopa* (with carbidopa); terbinafine cream + warfarin; warfarin + spironolactone
2 Detection of 20 major and 20 minor drug interactions in nine software systems* and five reference sources
3 Comparison of information provided for 20 major drug interactions in nine software systems* and five reference sources
AMH = Australian Medicines Handbook. MIMS DAI = MIMS DrugAlert Interactions. PI = product information. SIA = Stockley’s Interaction Alerts. SDI = Stockley’s Drug Interactions. * Software systems are de-identified and not displayed in the order listed in the text. Some systems did not detect all 20 interactions (see Box 2).
4 Recommendations for drug interaction alerts in prescribing and dispensing software
- 1. Sharma N, Kerr SJ, Whicker SD. Quality of drug interaction decision support prompts in prescribing software packages for Australian general practice. In: Proceedings of the Australian Health and Medical Research Congress; 2002 Nov 25–29; Melbourne, Vic. Sydney: AH&MR Congress, 2002.
- 2. Ahearn MD, Kerr SJ. General practitioners’ perceptions of the pharmaceutical decision-support tools in their prescribing software. Med J Aust 2003; 179: 34-37. <MJA full text>
- 3. National Prescribing Service. Knowledge, attitudes, awareness and behaviour of health professionals: National GP & Pharmacist Surveys. Sydney: NPS, 2006.
- 4. Ko Y, Abarca J, Malone DC, et al. Practitioners’ views on computerized drug–drug interaction alerts in the VA system. J Am Med Inform Assoc 2007; 14: 56-64.
- 5. Magnus D, Rodgers S, Avery AJ. GPs’ views on computerized drug interaction alerts: questionnaire survey. J Clin Pharm Ther 2002; 27: 377-382.
- 6. Spina JR, Glassman PA, Belperio P, et al. Clinical relevance of automated drug alerts from the perspective of medical providers. Am J Med Qual 2005; 20: 7-14.
- 7. van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13: 138-147.
- 8. Abarca J, Malone DC, Armstrong EP, et al. Concordance of severity ratings provided in four drug interaction compendia. J Am Pharm Assoc 2004; 44: 136-141.
- 9. Vitry AI. Comparative assessment of four drug interaction compendia. Br J Clin Pharmacol 2007; 63: 709-714.
- 10. Coiera EW, Westbrook JI. Should clinical software be regulated [editorial]? Med J Aust 2006; 184: 600-601. <MJA full text>
- 11. Beilby JJ, Duszynski AJ, Wilson A, Turnbull DA. Electronic decision support systems at point of care: trusting the deus ex machina [editorial]. Med J Aust 2005; 183: 99-100. <MJA full text>
- 12. van Roon EN, Flikweert S, le Comte M, et al. Clinical relevance of drug–drug interactions: a structured assessment procedure. Drug Saf 2005; 28: 1131-1139.
- 13. McInnes DK, Saltman DC, Kidd MR. General practitioners’ use of computers for prescribing and electronic health records: results from a national survey. Med J Aust 2006; 185: 88-91. <MJA full text>
- 14. Rossi S, editor. Australian medicines handbook [electronic version]. Adelaide: Australian Medicines Handbook, 2007. http://www.amh.net.au (accessed Sep–Nov 2006).
- 15. MIMS Australia. eMIMS version 5.00.0270. Sydney: CMPMedica Australia, 2006.
- 16. Baxter K, editor. Stockley’s drug interactions [electronic version]. London: Pharmaceutical Press, 2006. http://www.medicinescomplete.com (accessed Sep–Nov 2006).
- 17. Stockley Editorial Team. Stockley’s interaction alerts [electronic version]. London: Pharmaceutical Press, 2006. http://www.medicinescomplete.com (accessed Sep–Nov 2006).
- 18. Facts & comparisons 3.0. Drug interaction facts [electronic version]. St Louis, Mo: Facts & Comparisons, 2006. http://www.factsandcomparisons.com (accessed Sep–Nov 2006).
- 19. Pharmaceutical Pricing Section, Pharmaceutical Benefits Branch. Expenditure and prescriptions twelve months to 30 June 2005. http://www.health.gov.au/internet/main/Publishing.nsf/Content/3CC2D4DF821FE5ADCA2570F4000 5B9B1/$File/Full%20book%202004-05.pdf (accessed Aug 2006).
- 20. Barrons R. Evaluation of personal digital assistant software for drug interactions. Am J Health Syst Pharm 2004; 61: 380-385.
- 21. Perkins NA, Murphy JE, Malone DC, Armstrong EP. Performance of drug–drug interaction software for personal digital assistants. Ann Pharmacother 2006; 40: 850-855.
- 22. Abarca J, Colon LR, Wang VS, et al. Evaluation of the performance of drug–drug interaction screening software in community and hospital pharmacies. J Manag Care Pharm 2006; 12: 383-389.
- 23. Gaikwad R, Sketris I, Shepherd M, Duffy J. Evaluation of accuracy of drug interaction alerts triggered by two electronic medical record systems in primary healthcare. Health Informatics J 2007; 13: 163-177.
- 24. Bergk V, Gasse C, Schnell R, Haefeli WE. Requirements for a successful implementation of drug interaction information systems in general practice: results of a questionnaire survey in Germany. Eur J Clin Pharmacol 2004; 60: 595-602.
Publication of your online response is subject to the Medical Journal of Australia's editorial discretion. You will be notified by email within five working days should your response be accepted.