We need to chat about artificial intelligence

With the arrival of large language models such as ChatGPT, AI is reshaping how we work and interact

Long foretold and often dismissed, artificial intelligence (AI) is now reshaping how we work and interact as a society.1 For every claim that AI is overhyped and underperforming, only weeks or months seem to pass before a new breakthrough asks us to re-evaluate what is possible. Most recent, it is the very public arrival of large language models (LLMs) such as the generative pre-trained transformers (GPTs) in ChatGPT. In this perspective article, we explore the implications of this technology for health care and ask how ready the Australian health care system is to respond to the opportunities and risks that AI brings.

GPTs are a recent class of machine learning technology. Guided by humans who provide it with sample responses and feedback, ChatGPT was initially trained on 570 gigabytes of text, or about 385 million pages of Microsoft Word, and, at first release, the language model had 175 billion parameters.2 This massive model of the relationships between words is generative in that it produces new text, guided by the model, in response to prompts. It can answer questions, write songs, poems, essays, and software code. Other generative AIs such as DALL‐E, which is trained on images, can create startlingly good pictures, including fictitious or “deep fake” images of real people.3

Today's LLMs are story tellers, not truth tellers. They model how language is used to talk about the world, but at present they do not have models of the world itself. The sheer size of ChatGPT means that it can perform tasks it was not explicitly trained to do, such as translate between languages. ChatGPT amassed 100 million users in the first two months that it was available.4 So compelling are the linguistic skills of LLMs that some have come to believe such AI is sentient,5 despite the prevailing view that as statistical pattern generators, they cannot have consciousness or agency. Australian singer Nick Cave called ChatGPT “a grotesque mockery of what it is to be human” after seeing it generate new songs in his style.6

The health care uses of generative models will soon become clearer.7 Epic has agreed with Microsoft to incorporate its GPT‐4 model into their electronic health records, which have been used for over 305 million patients worldwide.8 LLMs are likely to find application in digital scribes, assisting clinicians to create health records by listening to conversations and creating summaries of the clinical content.9,10 They can create conversational agents, which change the way we search medical records and the internet, synthesising answers to our questions rather than retrieving a list of documents.11

We should prepare for a deluge of articles evaluating LLMs on tasks once reserved for humans, either being surprised by how well the technology performs or showcasing obvious limits because of the lack of a deep model of the world.12 Especially when it comes to clinical applications, producing text or images that are convincing is not the same as producing material that is correct, safe, and grounded in scientific evidence. For example, conversational agents can produce incorrect or inappropriate information that could delay patients seeking care, trigger self‐harm, or recommend inappropriate management.13 Generative AI may answer patients’ questions even if not specifically designed to do so. Yet all such concerns about technology limitations are hostage to progress. It would be foolish indeed to see today's performance of AI as anything other than a marker on the way to ever more powerful AI.

The unintended consequences of AI

It is the unintended consequences of these technologies that we are truly unprepared for. It was hard to imagine in the early innocent days of social media, which brought us the Arab Spring,14 just how quickly it would be weaponised. Algorithmic manipulation has turned social media into a tool for propagating false information, enough to swing the results of elections, create a global antivaccination movement, and fashion echo chambers that increasingly polarise society and mute real discourse.

Within two months of the release of ChatGPT, scientific journals were forced to issue policies on “non‐human authors” and whether AI can be used to help write articles.15 Universities and schools have banned its use in classrooms and educators scramble for new ways to assess students, including returning to pen and paper in exams.16 ChatGPT is apparently performing surprisingly well on questions found in medical exams.17

The major unintended consequences of generative models are still to be revealed.18 LLMs can produce compelling misinformation and will no doubt be used by malicious actors to further their aims. Public health strategies already must deal with online misinformation; for example, countering antivaccination messaging. Maliciously created surges of online messages during floods, heat events, and pandemics could trigger panic, swamp health services, and encourage behaviours that disrupt the mechanics of society.19

The national imperative to respond to the challenges of AI

With AI's many opportunities and risks, one would think the national gaze would be firmly fixed on it. However, Australia lags most developed nations in its engagement with AI in health care and has done so for many years.20 The policy space is embryonic, with focus mostly on limited safety regulation of AI embedded in clinical devices and avoidance of general purpose technologies such as ChatGPT. Some clinical colleges and organisations have been more “fleet of foot”, adapting their training programs or developing frameworks for the ethical use of AI.21,22 Yet there is currently no national framework for an AI‐ready workforce, overall regulation of safety, industry development, or targeted research investment. The national conversation on AI in health care has for now remained niche and low in priority.

Indeed, there has been a view in some quarters that all we need to do as a nation is adopt the best of what is produced internationally, and that we do not need deep sovereign capabilities. Nothing is further from the truth. Without some degree of algorithmic sovereignty (the capability to produce or modify AI in Australia), the nation is exposed to new risks and misses one of the most significant industrial revolutions of our times.23

We do not want to just export our clinical datasets and import back the models built with them. We should be a value adding economy that builds and exports these technologies ourselves. Australia's $1.4 billion clinical trials sector24 will face stiff international competition from those who use AI to identify, enrol and monitor patients more effectively and at a lower cost. Our health response to climate change will depend heavily on digital health and AI for mitigation and adaptation.25 Further, AI requires local customisation to support local practices and reflect diverse populations or health service differences.9 Without local capability, paying to modify clinical AI will likely become a huge burden on our health system. Critically, using AI requires retraining of the workforce, retooling health services, and transforming workflows. The health system is already resource‐constrained, and such changes will not happen without strategic investment.

The national discussion on what to do next has begun, with a roadmap for AI in health care produced by the Australian Alliance for AI in Healthcare (AAAiH) — a national collective of over 100 organisations including academia, industry, peak bodies, and health services providers.26 The roadmap is the product of feedback from 152 key stakeholder organisations and individuals and contains 24 recommendations across eight priority areas. The highest community priority identified was for health care AI to be safe for patients and developed and used ethically. AI privacy and data security were another major concern. Respondents identified the need for genuine whole‐of‐nation leadership in the health care AI space as well as robust governance of the sector. Gaps in our workforce capability to build and use health care AI were clearly identified, as was the need for consumers to be fully engaged in shaping the health care AI agenda. Respondents also rated the gaps in our capability to adopt AI into practice and the need to enhance local industry capability as issues needing clear attention. While a great start, the roadmap now needs to be converted into action, and that will require bringing together the skills and interests of many stakeholders, from government, consumer bodies, clinicians, industry, health service providers, and academia.

We can only expect the pace of AI innovation to accelerate, and for its consequences, good and ill, to multiply. We have a national imperative to both harness and benefit from these technologies, and not be hostages to the decisions of others. The time for urgent national engagement has arrived.