Show me the money: how do we justify spending health care dollars on digital health?

Leanna Woods, Rebekah Eden, Oliver J Canfell, Kim‐Huong Nguyen, Tracy Comans and Clair Sullivan
Med J Aust 2023; 218 (2): 53-57. || doi: 10.5694/mja2.51799
Published online: 12 December 2022

Focusing solely on financial measures is unlikely to deliver a comprehensive view of the value of digital health

Digital health, which refers to the use of digital technology to provide and support health care services, promises to strengthen health systems worldwide and has been accelerated by the coronavirus disease 2019 (COVID‐19) pandemic.1 Amid the rapid digital transformation of health care,2,3 the value of sizeable digital health investments remains unclear.3,4 The investments required for digital health are often substantial and may come at a cost to existing health care delivery models. Decision makers can be paralysed by a situation where investment in digital health is unavoidable but the conventionally measured short term outcomes often do not provide a convincing fiscal return on investment.2,3

However, health systems are now shifting from perceiving digital health as a cost that needs to be recouped to a quality improvement tool that can positively transform health care. This is because large‐scale digital health implementations, such as electronic medical records (EMRs), provide significant quality and safety benefits, including:5

  • reduced unwarranted variation in care;
  • reduced preventable harm;
  • improved patient centredness; and
  • enhanced opportunities for monitoring, risk management, and quality improvement.

These quality and safety benefits, despite being significant and meaningful to clinicians and consumers, are difficult to cost, rarely deliver a rapid financial benefit to the funder, and are traditionally not measured in financial evaluations.

To maximise the potential of digital health, a more sophisticated approach than simple monetary return on investment needs to be developed and adopted. The quadruple aim of health care — better population health outcomes, improved care experience for patients, improved work life of health professionals, and reduced health care costs6 — is a straightforward and increasingly adopted approach for evaluating quality improvement interventions in health care.

In this perspective article, we show the potential of the quadruple aim to transform how digital health is measured and so how it is valued. Using the quadruple aim as a streamlined, scalable framework to quantify the real‐world impact of digital transformations allows a new level of granularity that enables continuous and clinically meaningful improvement. We present a case study of applying the quadruple aim framework to understand the value of a hospital‐based EMR, delivering 65% of all public health care across 16 digital hospitals in Queensland, Australia.

Measuring health system performance: from business cases to the quadruple aim

Criteria for evaluating the performance of health care have evolved since The world health report 2000 initially identified three fundamental criteria to measure the performance of health systems: good population health, the ability to respond to consumers’ expectations, and fairness of financial contribution.7 In 2001, the Institute of Medicine published consensus‐based measures to evaluate functioning health care systems that are safe, effective, patient‐centred, timely, efficient and equitable.8 Then in 2008, the Institute for Healthcare Improvement9 proposed pursuit of the triple aim: improving patient experience, improving health, and reducing the cost of care. This was later challenged in an article published in 2014 which argued that positive engagement of the health workforce was paramount to achieving the triple aim.6 The fourth dimension — improving the experience of the health care workforce — was added. The quadruple aim has been adopted in health care workforce,10,11 innovation implementation,12 and COVID‐19 pandemic13 contexts to drive a balanced scorecard for health care projects, forcing health care funders to look beyond traditional fiscal return. The discourse has evolved in 2022 to include health equity to form the quintuple aim.14

Existing digital health evaluation models

Recent systematic reviews have highlighted multiple strategies to assess the impact of digital health to inform future investments15,16 and have reported challenges due to the heterogenous, emerging and uncertain nature of digital health impacts.15,17,18 A range of models are currently in use to evaluate digital health (Box 1).

Information technologies (eg, computers) in health care have been implemented for decades, but advances in health care technologies and the data they produce (eg, mobile health, virtual care, precision medicine) are transforming care. Recouping the costs from an information technology investment in the short to medium term is unlikely due to the large upfront expense and the limited efficiencies that can be delivered in the short term. The traditional business case approach, which many existing digital health evaluation models adopt, fails to consider the value of the downstream effects of a connected digital health ecosystem, which enables sophisticated technological advances such as artificial intelligence, machine learning and precision medicine.

We suggest that digital health is no longer a technical capability for efficiency, but rather a critical enabler to achieving the quadruple aim. With the burden on the health care system to continue service delivery with static resourcing, an ageing population and growing health inequities, digital health provides an unrivalled opportunity to improve care at scale. The quadruple aim balances the economic costs (input side) with clinician experience (throughput) and health care quality and consumer experience (outputs).6 This is critical in the area of digital health, where a solitary focus on the deployment of health care technology has resulted in negative experiences and, in some cases, harm.22 Investing in digital health needs careful consideration, as technological advancements may cause unanticipated consequences, such as creating a digital divide,23 the depersonalisation of the clinician–patient relationship,24 poor integration with other health care systems,1,24 and increased clinician workload.1 Modernising the frame of reference we use for assessing digital health impacts is critical and needs to be aligned to modern health care delivery underpinned by the quadruple aim.

Case study: use of the quadruple aim for evaluating EMR implementations

Digitally mature health services are exploiting the capabilities of EMRs as a quality improvement platform, rather than solely information technology. Investments in hospital‐based EMRs are controversial, as they disrupt usual ways of working, can create slower workflows, and have been shown to contribute to clinician frustration and burnout.24,25 Evidence supports the use of EMR capabilities such as clinical decision support systems, electronic medication management and digital clinical care pathways to improve care.15,26 These capabilities are driven by the secondary use of routinely collected clinical data to improve the quality and safety of health care delivery, which has become an important enabler for health care improvement.27

The impact of EMR implementations over time can be described using a three‐horizon conceptual model of digital transformation (Box 2):

  • horizon 1 focuses on building digital workflows to streamline the collection of clinical data;
  • horizon 2 creates aggregated data and analytics for quality improvement; and
  • horizon 3 implements new digitally enabled models of care.

Traditional business cases evaluating the impact of hospital‐based EMRs only evaluate horizon 1 (the implementation of the digital workflows), neglecting the value of the aggregated data, analytics and the new models of care that develop over time.28

The immediate implementation impact is limited to easily measurable efficiencies (eg, reduction in printing costs).28 The increase in quality and safety enabled by data, analytics and new models of care in a digitally transformed health care system are realised in horizons 2 and 3, which are not captured in traditional business cases. We are therefore not adequately measuring the real long term impact of EMRs on health, patients, clinicians, health services, and our community.

In our case study, we also sought to understand the economic value of a hospital‐based EMR implementation. Economic value refers to the value an individual places (from both subjective and objective elements) on a good or service based on the benefit (perceived or actual) that they derive from it. It can be difficult to accurately measure the economic value of health care, where subjective constructs such as quality of life and patient experience are important elements.

A new framework: a balanced view of digital health value

We previously undertook a comprehensive literature review to extract published metrics and evaluation methods for EMR implementations,28 and mapped these to the quadruple aim of health care. We then improved the evaluation framework, adapting existing economic approaches in other sectors and implementing a process of engagements including consultations with stakeholders across academia, government and industry.

The new framework we developed incorporates additional elements of value that do not yet have a market price but have a definite economic value, whether positive or negative, for the stakeholders (Box 3). This includes items such as workforce satisfaction, which can be negative in the short term due to disruption and positive in the long term due to learned improved work practices. The framework captures elements that are hypothesised to create benefits but require testing for confirmation (eg, savings from reduced primary care use after admission to hospital). Additional elements will emerge over time through system improvements, new technologies and evolving models of care.


Health care organisations are currently not measuring the true value of digital health implementations and, thus, digital investment may not be prioritised given the competing health care demands. A modern, streamlined framework is critical to providing a balanced and long term view of the impact of digital transformation as it shifts away from an efficiency tool to a quality and safety improvement platform that enables innovation.

The impact of digital health interventions that are meaningful to staff, clinicians and consumers that goes beyond simplistic profit and loss evaluations needs to be acknowledged and measured. Focusing on financial outcomes alone is unlikely to deliver a comprehensive view of the value of digital health and will slow the necessary transformation of our increasingly unsustainable health care system. Clinicians should be aware that there is a balanced way to assess both positive and negative impacts of digital health implementations. This ensures that, in addition to productivity, non‐financial benefits such as quality of care, clinician experience and health outcomes are at the centre of the inevitable digital transformation of health care.

Box 1 – Sample of digital health evaluation methods in current use

Existing approach




Economic evaluations (eg, return on investment)4,19

Benefit minus the cost expressed as a proportion of the cost

Value the financial return of a business investment

Focus on financials only; limited to comparison to existing services4

Canada Health Infoway's Benefits Evaluation Framework18

Proposes that technical attributes of the system (ie, system, information and service quality) influence how the system is used and staff satisfaction with the system, which in turn influences the benefits attained

Comprehensive measures evaluating benefits of digital health linked to quality of outcomes, patients’ access to services, and productivity improvements

Workforce implications are overlooked

Electronic Medical Record Adoption Model, Healthcare Information and Management Systems Society20

Evaluates the maturity of an organisation based on the extensiveness of their digital health investments

Provides a tool for organisations to benchmark their digital capability

Focus on technology with limited incorporation of organisational and human factors21


Box 2 – Identified impacts of electronic medical records over a 10‐year time frame


Box 3 – Mapping the quadruple aim of health care to impacts of the electronic medical record (EMR)

KPI = key performance indicators; QALY = quality‐adjusted life‐years.

Provenance: Not commissioned; externally peer reviewed.

  • Leanna Woods1,2,3
  • Rebekah Eden4
  • Oliver J Canfell1,2,3,5
  • Kim‐Huong Nguyen1,6
  • Tracy Comans1
  • Clair Sullivan1,2,7

  • 1 Centre for Health Services Research, University of Queensland, Brisbane, QLD
  • 2 Queensland Digital Health Centre, University of Queensland, Brisbane, QLD
  • 3 Digital Health Cooperative Research Centre, Sydney, NSW
  • 4 Queensland University of Technology, Brisbane, QLD
  • 5 University of Queensland, Brisbane, QLD
  • 6 Global Brain Health Institute, Trinity College Dublin and University California, San Francisco, Dublin, Ireland
  • 7 Royal Brisbane and Women's Hospital, Metro North Hospital and Health Service, Brisbane, QLD


Open access

Open access publishing facilitated by The University of Queensland, as part of the Wiley ‐ The University of Queensland agreement via the Council of Australian University Librarians.


We thank Queensland Health staff for their contribution to this work. Leanna Woods and Oliver Canfell are supported by the Digital Health CRC, which is funded by the Commonwealth Government through the Cooperative Research Centres Program. The funding source provides salary support only, and they had no input in the data collection, analysis, interpretation or reporting.

Competing interests:

No relevant disclosures.

  • 1. World Health Organization. Recommendations on digital interventions for health system strengthening. WHO, 2019. (viewed Nov 2022).
  • 2. Rahimi K. Digital health and the elusive quest for cost savings. Lancet Digit Health 2019; 1: e108‐e109.
  • 3. Greenhalgh T, Wherton J, Papoutsi C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale‐up, spread, and sustainability of health and care technologies. J Med Internet Res 2017; 19: e367.
  • 4. World Health Organization. Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. WHO, 2016. (viewed Nov 2022).
  • 5. Shaw T, Hines M, Kielly‐Carroll C. Impact of digital health on the safety and quality of health care. ACSQHC, 2018.‐and‐resources/resource‐library/impact‐digital‐health‐safety‐and‐quality‐health‐care (viewed Nov 2022).
  • 6. Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014; 12: 573‐576.
  • 7. World Health Organization. The world health report 2000: health systems: improving performance. WHO, 2000. (viewed Nov 2022).
  • 8. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington (DC): National Academy Press, 2001.
  • 9. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff (Millwood) 2008; 27: 759‐769.
  • 10. Rathert C, Williams ES, Linhart H. Evidence for the quadruple aim: a systematic review of the literature on physician burnout and patient outcomes. Medical Care 2018; 56: 976‐984.
  • 11. Privitera MR. Addressing human factors in burnout and the delivery of healthcare: quality and safety imperative of the quadruple aim. Health 2018; 10: 629‐644.
  • 12. Liddy C, Keely E. Using the quadruple aim framework to measure impact of heath technology implementation: a case study of eConsult. J Am Board Fam Med 2018; 31: 445‐455.
  • 13. Wong AH, Ahmed RA, Ray JM, et al. Supporting the quadruple aim using simulation and human factors during COVID‐19 care. Am J Med Qual 2021; 36: 73‐83.
  • 14. Nundy S, Cooper LA, Mate KS. The quintuple aim for health care improvement: a new imperative to advance health equity. JAMA 2022; 327: 521‐522.
  • 15. Eden R, Burton‐Jones A, Scott I, et al. Effects of eHealth on hospital practice: synthesis of the current literature. Aust Health Rev 2018; 42: 568‐578.
  • 16. Keasberry J, Scott IA, Sullivan C, et al. Going digital: a narrative overview of the clinical and organisational impacts of eHealth technologies in hospital practice. Aust Health Rev 2017; 41: 646‐664.
  • 17. Eden R, Burton‐Jones A, Staib A, Sullivan C. Surveying perceptions of the early impacts of an integrated electronic medical record across a hospital and healthcare service. Aust Health Rev 2020; 44: 690‐698.
  • 18. Pita‐Barros P, Bourek A, Brouwer W, Lehtonen L. Assessing the impact of digital transformation of health services — report of the Expert Panel on effective ways of investing in Health (EXPH). Luxembourg: Publications Office of the European Union, 2019.‐11/022_digitaltransformation_en_0.pdf (viewed Nov 2022).
  • 19. Masters R, Anwar E, Collins B, et al. Return on investment of public health interventions: a systematic review. J Epidemiol Community Health 2017; 71: 827‐834.
  • 20. Healthcare Information and Management Systems Society. Electronic Medical Record Adoption Model (EMRAM). Chicago (IL): HIMSS Analytics (viewed Mar 2021).
  • 21. Cresswell K, Sheikh A, Krasuska M, et al. Reconceptualising the digital maturity of health systems. Lancet Digit Health 2019; 1: e200‐e201.
  • 22. Marwaha JS, Landman AB, Brat GA, et al. Deploying digital health tools within large, complex health systems: key considerations for adoption and implementation. NPJ Digit Med 2022; 5: 13.
  • 23. Clark CR, Akdas Y, Wilkins CH, et al. TechQuity is an imperative for health and technology business: Let's work together to achieve it. J Am Med Inform Assoc 2021; 28: 2013‐2016.
  • 24. Alami H, Lehoux P, Gagnon MP, et al. Rethinking the electronic health record through the quadruple aim: time to align its value with the health system. BMC Med Inform Decis Mak 2020; 20: 32.
  • 25. Downing NL, Bates DW, Longhurst CA. Physician burnout in the electronic health record era: are we ignoring the real cause? Ann Intern Med 2018; 169: 50‐51.
  • 26. Hodgson T, Burton‐Jones A, Donovan R, et al. The role of electronic medical records in reducing unwarranted clinical variation in acute health care: systematic review. JMIR Med Inform 2021; 9: e30432.
  • 27. Barnett A, Winning M, Canaris S, et al. Digital transformation of hospital quality and safety: real‐time data for real‐time action. Aust Health Rev 2018; 43: 656‐661.
  • 28. Nguyen KH, Wright C, Simpson D, et al. Economic evaluation and analyses of hospital‐based electronic medical records (EMRs): a scoping review of international literature. NPJ Digit Med 2022; 5: 29.


remove_circle_outline Delete Author
add_circle_outline Add Author

Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Online responses are no longer available. Please refer to our instructions for authors page for more information.