To the Editor: I was interested to read the results of Sweidan and colleagues’ study of drug interaction alerts in prescribing and dispensing software.1 I believe their use of the terms “sensitivity” and “specificity” differ from the standard definitions, which are usually:

To test sensitivity and specificity, one requires a dataset that includes positives and negatives. I do not view “minor interactions” as a complete set of negatives, because a complete set of negatives should include a statistically valid number of randomly chosen drug sets without interactions. Minor interactions do not meet my criteria for “negatives” because, to me, a minor interaction is still an interaction that may sometimes be clinically significant. A true negative should meet the test of “never clinically significant”. Some reported minor interactions would meet that test and some would not. I believe sensitivity and specificity data should be reported for both major and minor interaction alerts.

I also believe there should be some alignment of definitions between “drug interaction” research and “adverse drug event” research.3 Bates and colleagues talked about “adverse drug events” and “preventable adverse drug events” in 1995.4 Of most clinical interest are the preventable adverse drug events, which could be minimised by the use of appropriate decision support.5,6

Certainly, there is a need for independent assessment of the quality of electronic prescribing decision support systems. A robust assessment methodology is required to permit potential government regulation of such resources, which are of national and community importance.

To the Editor: As the former Clinical Information Specialist Manager for the MIMS DrugAlert knowledgebase (from 2003 to 2005), I write in response to the study by Sweidan and colleagues examining the quality of drug interaction alerts in prescribing and dispensing software.1

The authors point out that the success of any knowledgebase in providing clear, correct and specific alerts at the point of care is subject to the quality of its technical integration into decision support software. I would like to add that the sensitivity of drug interaction decision support is determined largely by the knowledgebase, while the specificity of the system is subject to the intelligence of the software in which it is employed. I would be interested to know if the low specificity that Sweidan et al found for the MIMS DrugAlert database was due to lack of use of the severity or level of evidence settings, or having these set at inappropriate levels.

The MIMS DrugAlert knowledgebase was in some ways a unique decision support database, written by Australian staff for use in Australia and New Zealand. It soon became one of the largest commercially available databases of drug interaction information in the world, covering over 4600 drug-class and individual drug interactions. Its writing alone was a remarkable feat, being completed in a matter of months and further expanded over a subsequent 18-month period. There are significant variations in practical advice between American and European sources of drug interaction information. In writing the MIMS DrugAlert database, we sought to communicate “the right information, at the right time, in the right way” to local professionals.

The foundations of MIMS DrugAlert were based on a clear understanding that we would be representing relatively simple pharmacological principles through the structure and content of a relational database. This meant creating interacting drug classes reflective of the pharmacological properties of groups of drugs, rather than simply grouping drugs based on their chemical families alone.

Sweidan and colleagues should be congratulated on highlighting the need for comprehensive, accurate and useful information that can reduce medication error and save lives at the point of care. What is lacking is a clinical outcomes-based study focusing on the real-world benefits that can be achieved if the right system can be implemented in the right way, at the right price. Perhaps this type of research would then build on the excellent, basic foundational research carried out by Sweidan et al.

In reply: Cheong notes that the terms “sensitivity” and “specificity” have a slightly different meaning in our study compared with the usual definitions. This was intentional, and the definitions we used are clearly described in our article.1

Our definitions for sensitivity and specificity were based on two important practical considerations. First, an electronic prescribing system should alert the clinician to potentially clinically significant drug interactions (“true positives” by our definition); and second, a system should not inundate clinicians with alerts containing irrelevant or unhelpful information about minor or clinically unimportant interactions (“false positives” by our definition). We know that these latter alerts can cause “alert fatigue” and are a subject of complaint for doctors and pharmacists. We are not aware of there being any problem with prescribing systems producing alerts for pairs of drugs that do not interact at all; hence, we did not investigate this group.

Tan questions whether the low specificity we found for the MIMS DrugAlert database may have been due to inappropriate severity level settings. Although it might seem appealing to reduce the number of alerts by allowing users to “switch off” drug interaction alerts that are classified as low severity, there are difficulties in doing this because of a lack of evidence for the application of severity ratings to drug interactions. Severity ratings are subjective — studies have shown there is little consensus on such ratings between major reference sources.2,3 This is not surprising, given that there is little evidence available on adverse clinical outcomes resulting from drug interactions, and also because the clinical outcome is context-dependent according to variables such as patient characteristics and drug dosage. We believe that, rather than relying on software vendors or users to switch off some alerts, drug interaction knowledgebases should include only potentially clinically significant interactions.

We tested all systems at the lowest severity setting (if available) for consistency, and to maximise detection of drug interaction alerts. For minor interactions, the rating was based on both presence of the alert and quality of the information — if a minor interaction was either not detected or was detected and provided appropriate information indicating it was minor, then it was a “pass”.