Will online symptom checkers improve health care in Australia?

The available tools are largely unregulated, and do not reliably guide people to the right care at the right time

In times when health services are under increasing strain, digital health technologies such as online symptom checkers appear convenient and cost‐effective tools for reducing the burden on clinics, telemedicine services, and emergency departments. In practical terms, an online symptom checker is a smartphone app or web‐based form that can provide a diagnosis on the basis of a set of self‐reported symptoms. They can suggest diagnoses for a broad range of conditions with which people may present to a clinic or emergency department. When they work properly, symptom checkers should turn current practice guidelines into tools that can diagnose and triage patients at low cost.

In this issue of the MJA, Hill and colleagues1 used a vignette‐based approach to evaluate the ability of 36 symptom checkers available in Australia to correctly diagnose and triage people. The vignettes included scenarios that health providers encounter in Australia, including shingles, heart attacks, and viral upper respiratory infections. The authors found that the first suggested diagnoses were correct for about one‐third of the vignettes, and that triage advice was appropriate for half. The best performing tool for triage advice was the Australian Healthdirect symptom checker, which provided correct advice for 28 of 46 vignettes.

What makes a symptom checker safe and useful?

Symptom checkers claiming to use artificial intelligence (AI) methods outperformed other checkers in the vignette‐based analysis.1 But characteristics other than computational sophistication also influence the safety and utility of symptom checkers, including updatability, localisation, failing gracefully, and intent.

Symptom checkers should be updatable, easily and frequently, in order to incorporate new evidence. In a public health crisis, decisions about where and how to seek care can change daily. When looking online for symptom checkers to include in their study, Hill and colleagues1 searched, among other things, for flu‐like symptoms (“cough, fever, headache”). It is safe to assume that none of the included symptom checkers would have been equipped to keep up with the emerging COVID‐19 pandemic. Also largely missing was localised advice: the capacity to recommend care pathways aligned with local practice and informed by regional differences in disease burden.

In software engineering, the concept of failing gracefully describes the ability of programs to detect and explain or resolve functional failures as they occur. While health care providers misdiagnose about 5% of patients, symptom checkers do so in 39–88% of cases.2 Unlike symptom checkers, however, health care providers can manage uncertainty, saying “I don't know”, and seeking further information.

Intent is perhaps the most challenging aspect of symptom checkers to assess, but it can lead to serious problems. When Babylon Health evaluated their own AI‐based triage system, they claimed that the accuracy of diagnosis was “comparable to human doctors”,3 a claim met with substantial criticism when independently scrutinised.2 In Australia, the provider of an appointment booking app was found to be in partnership with a law firm that targeted advertising to users who reported trauma injuries.4 Identifying and mitigating the effects of financial conflicts of interest related to symptom checkers is an unresolved challenge.

Should we be integrating online symptom checkers into clinical practice?

People see value in symptom checkers,5 but while self‐reported outcomes are important, they are not necessarily associated with safety or improved health service efficiency.

To be meaningfully integrated into clinical practice, decision support tools should be approved by a national governing body and subject to ongoing surveillance, similar to drugs and medical devices. Community tools such as symptom checkers are typically not subject to the same level of regulation,6 although software for clinical diagnosis is regulated overseas to some degree.7 Most symptom checkers avoid regulation by including a disclaimer that they should not be used to replace expert medical advice.

Another important finding by Hill and her co‐authors1 is that symptom checkers are generally risk‐averse, recommending more urgent care than needed, echoing the findings of overseas analyses.8,9 Nevertheless, safety problems that can arise when less urgent care is recommended for serious conditions should not be dismissed, even if such cases are rare.6

Despite knowing how well a broad range of symptom checkers perform in general, we are not yet ready to approve and deploy them in the community. We need to evaluate specific checkers against a broader set of criteria in a formal approval process, as is required for other diagnostic tools and interventions.7 After approval, ongoing and transparent surveillance of outcomes by users10 could provide a better basis for integrating symptom checkers into the health system.