In this issue of the Journal, a study from the National Prescribing Service (NPS) examines the quality of drug interaction alerts generated by nine clinical software systems currently used by general practitioners and pharmacists in Australia for prescribing or dispensing medications (Sweidan et al).1 The findings will come as no surprise to those who have repeatedly expressed concern about the shortcomings of clinical decision support software.2,3 Only half of the six prescribing systems examined by the NPS alerted users to all 20 of the major drug–drug interactions tested, which can occur with commonly used drugs and with the potential to trigger serious adverse reactions. The best of the three dispensing systems detected 19 of these drug interactions. Yet Australian GPs are heavily reliant on such software alerts: 88% of respondents to a recent national survey reported relying on their prescribing software to check for drug–drug interactions.4 Any failure of decision support systems to provide adequate drug safety alerts is thus likely to pose risks to patient safety.

The NPS study provides a snapshot of software performance in response to artificially generated test cases. Different test cases may have yielded different results. Further, as valuable as they are, such studies cannot provide data on the cause of these failures or the likelihood that they will ultimately result in medication errors.

At a fundamental level, we know that the performance of any decision support system will be determined by the completeness and accuracy of its knowledgebase. A second potential cause of missed alerts is the internal procedures and logic used within a decision support system. For example, even though the MIMS DrugAlert Interactions knowledgebase detects all 20 major drug interactions tested in the NPS study, the four prescribing systems based on this knowledgebase failed to uniformly report all of these alerts.1 Anecdotally, we also know that there is variation in the logic used to determine which drug interactions different manufacturers elect to display, and which are treated as low priority. At present, there is scarce information available to indicate which of these different components of decision support systems is most likely to generate safety problems. As a consequence, it is currently not possible to provide guidance to clinicians, policymakers or system manufacturers on the most appropriate safety practices needed to avoid misadventure.

Moving beyond laboratory testing of software, there is a critical need to examine the safety of decision support systems in the hands of typical users. There is growing evidence that busy clinicians routinely disable or override computer advice. “Alert fatigue” is a well known consequence of using systems that generate high rates of non-serious or irrelevant alerts.5 The long-term impact of using such systems and their influence on clinicians’ decision making have yet to be systematically investigated. Indeed, the NPS found that the three prescribing systems that alerted users to all the major, clinically significant drug interactions also generated unhelpful alerts for 30%–55% of the clinically unimportant interactions.1 To reduce alert fatigue, software designers could consider smarter decision support systems that can be trained to meet individual practice requirements (akin to an email spam filter that can be gradually trained to recognise and remove irrelevant messages). For instance, GPs could train their prescribing software to provide alerts only for newly prescribed medications and ignore repeat medications where the clinician has previously noted an alert.

This lack of clear evidence about the causation of computer-related failures in decision support systems, either on their own or in the hands of typical users, is likely to hamper international efforts to improve the safety governance of clinical software. Among the efforts seeking to address calls to regulate the safety of decision support systems,6 the United States Certification Commission for Healthcare Information Technology is introducing specific requirements for drug interaction alerts in ambulatory care systems.7 In September 2008, the United Kingdom’s National Health Service, which took a lead role in embedding a safety management approach into its procurement processes, published simple developer guidelines for safety features in prescribing systems.8 Future versions are planned to cover drug–drug interaction checking and other decision support functions not included in the initial specification. The International Organization for Standardization, using a risk-management approach, is developing standards for the construction, implementation and use of clinical software. In contrast, even though it is clear that we do need to move to some form of decision support system accreditation in Australia, safety governance of software does not yet seem to be on the agendas of the National E-Health Transition Authority or the Therapeutic Goods Administration.

As we approach 5 years since publication of the first study to measure deficiencies in safety features of clinical software,9 research efforts have not translated into changes in clinical software governance in Australia. There is little local support for this research, and examination of the safety of clinical software has not moved beyond the use of artificial test cases. Yet it is clear that the safety of clinical software is as much a product of the human user as it is of the machine. If we are to move from safe design to safe use of clinical software, we must accelerate our efforts to systematically examine the safety of decision support systems in the hands of users. While no one suggests that clinicians should stop using decision support systems, given their clear benefit to patients, we need to understand the “side effects” or unanticipated consequences of the technology,10 and understand in which situations their use might lead to unacceptable clinical risk.