Developing clinical indicators for oncology: the inaugural cancer care indicator set for the Australian Council on Healthcare Standards

There is a significant growth in the number of patients being diagnosed with cancer due to the ageing population, improved and earlier cancer detection and rising incidence.1 Concurrently, patients are receiving efficacious treatment for longer periods, spawning the once unthinkable concept of cancer as a chronic disease. Furthermore, the increasing cure rate has given rise to new areas of care such as cancer survivorship.2 Innovation in all aspects of cancer care has led to paradigm shifts in approach, with a patient‐centred model of multidisciplinary care now being the international gold standard.

The provision of cancer care is rooted in evidence‐based practice, and new diagnostic and therapeutic options are growing rapidly. Coupled with this expansion is a rise in patient and consumer advocacy and cancer health literacy, with patients and societies demanding optimal and equitable care.3 This demand extends beyond technical expertise to encompass excellence in patient experience.4 Hence, there is a pressing need for systems of transparent and accessible accountability for cancer care from individual and institutional health care providers.5

Clinical indicators are tools that provide methodologically consistent comparators of service provision across institutions and within various jurisdictions and/or defined subpopulations.6 Their purpose, along with an associated standards program is to “assist healthcare organisations to identify areas for improvement”.7 They comprise one of a suite of assessment tools measuring whether a standard in patient care is being met, which can then provide evidence for accreditation. The definition and implementation of performance indicators promote structure, organisation and a common language in reporting processes. However, clinical indicators do not provide answers per se; rather, they are designed to flag potential problems by identifying variations within or between health services. Clinical indicators can be part of a system designed to close such gaps as well as to improve outcomes for all.8,9,10

Clinical indicator sets are groups of measures developed with plans and frameworks that are used iteratively to compare metrics over time and according to various demographic‐ and disease‐related factors.11 Of particular concern is the lack of equity of access for vulnerable populations. In Australia, cancer outcomes are documented to be worse for people of Aboriginal and Torres Strait Islander origin, culturally and linguistically diverse groups, older people, and people of low socio‐economic status.12 Challenges in developing universal clinical indicators for cancer care in Australia lie in the multiple, intertwining pathways from diagnosis through to treatment and then follow‐up. Almost every patient interacts with many providers across both public (government‐funded) and private (some government funding, health care insurance and/or self‐funded) systems. In addition, patients access both community‐based care, overseen by the federal health system, and hospital‐based care, administered by state governments. Cancer care continues for a prolonged period for increasing numbers of patients, lasting many years for the majority, with the added challenges of patients relocating and changing practitioners during longer time frames. Thus, the diversity of settings poses a significant challenge for the development of a clinical indicator.13

The Australian Council on Healthcare Standards (ACHS) is an independent, not‐for‐profit accreditation agency with over 800 member health care organisations across all Australian states and territories, as well as internationally in Asia and the Middle East. The Clinical Indicator Program, established in 1989, is an ACHS data collection and reporting service that measures and benchmarks performance of aspects of clinical care in peer organisations, with the motto “If you can’t measure it, you can’t manage it”. Each clinical indicator set has a user manual containing an introduction to the set, stratification variables, reporting periods, and applicable clinical codes. For each individual indicator, the rationale, definition of terms, inclusion and exclusion criteria, data cleaning rules, numerator, denominator and literature‐based evidence are provided. Data are submitted through an online performance indicator reporting tool, aggregated and analysed. Comparative reports are provided biannually to submitters, with public industry‐wide summaries published annually via the Australasian Clinical Indicator Report.14 There are currently 21 ACHS indicator sets, comprising over 338 indicators across a broad range of health care generic and specialist areas and settings, providing the largest dedicated clinical indicator data collection and reporting service globally.

The process to develop a cancer clinical indicator set originated from requests from ACHS member organisations. ACHS initially developed clinical indicators for radiation oncology in 1999 — this set is now in its fifth iteration and data have been annually provided by up to 20 health care organisations. Based on its success, ACHS recognised a need for high level, overarching clinical indicators to measure other key components of the cancer journey.

Methods

The ACHS Performance and Outcomes Service supervised a four‐stage process based on a modified Delphi approach. This is a widely used methodology for establishing guidelines on standard practice of care or quality indicators, involving a series of discussion and ranking rounds with selected experts aimed at achieving consensus through an iterative process.15,16,17 The direction, name and area of potential indicators were developed in stage 1. This involved the formation of a 16‐member steering committee of key experts in cancer treatment, policy, nursing, outpatient care, radiation oncology and consumer advocacy, representing diverse experience and perspectives. The experts were nominated by various national bodies, to ensure appropriate representation from relevant societies, colleges, associations and organisations as well as consumer and cancer advocacy groups. The Chair was nominated by the Clinical Oncology Society of Australia (COSA). Members represented Cancer Council Australia, the Cancer Institute New South Wales, the Royal Australian and New Zealand College of Radiologists, the Medical Oncology Group of Australia, the Cancer Nurses Society of Australia, the Royal Australasian College of Medical Administrators, the Society of Hospital Pharmacists of Australia, the Australian Private Hospitals Association, the Peter MacCallum Cancer Centre (representing integrated cancer centres) and Icon Cancer Care (representing private oncology specialty providers).

At a full day, face‐to‐face meeting, the committee scoped the project and guided the terms for a comprehensive literature review to identify and list measures used by various international groups for assessment of quality care for cancer. There was agreement not to overlap with any existing clinical indicators within other sets, including the long‐standing radiation oncology clinical indicator set, based on the rationale that these are mature sets with ongoing measurements. However, it was acknowledged that the sets may be combined in the future. The steering group also recognised the need to respect the various existing Australian oncology quality frameworks, such as the Victorian cancer plan monitoring and evaluation framework, noting that the presence of various state‐based systems may pose challenges due to lack of uniformity.

Stage 2 involved a literature review and a systematic search of global cancer societies for scales and indicators currently being used. The steering committee then undertook an iterative process to review the list of extracted candidate indicators by scoring and prioritising each indicator through an online survey that took approximately 3 hours to complete. Individual indicators were ranked from 0 to 5 (lowest to highest priority) for each of two criteria: ease of accessibility and collectability of the data and clinical relevance (including potential actionability in terms of quality improvement and assessment of best practice performance). From this, a priority list of potential clinical indicators was constructed.

Each of the top ranked indicators emerging from stage 2 was discussed in detail before being accepted, rejected or modified in stage 3. Importantly, the wording and measurement tool for each indicator was optimised. Following this, a second full day, face‐to‐face meeting with a larger reference group of 20 stakeholders, including policy experts, key opinion leaders from a range of specialties and major cancer service providers, nursing members, representation from metropolitan and rural services, pharmacists, statisticians, indicator specialists, and community members.

In stage 4, the steering committee oversaw the construction of the cancer care user manual, with ACHS facilitating the clinical coding on the new indicators. The manual was endorsed by COSA then ratified by the ACHS Board of Directors, after which the document became live on the website (www.achs.org.au/programs-services/clinical-indicator-program).

Recommendations

The issues considered in framing the oncology clinical indicator set, formulated in the initial full day, face‐to‐face workshop of the steering committee, are summarised in Box 1. Additional stakeholders to invite to the second full day meeting to resolve the final indicator set were also suggested.

In stage 2, the literature review and search of metrics used by international accreditation and cancer societies identified or resolved 65 potential indicators across nine domains (Supporting Information, table 1). These were reviewed electronically, with each indicator scored by steering committee members for both ease of collection and clinical relevance. This process generated a priority list of 27 indicators, streamlined into six domains, with four additional candidates (Supporting Information, table 2). Each indicator was then debated at the second full day meeting, attended by the broader stakeholder group to resolve the final set (Supporting Information, table 3).

The process ultimately resolved 16 cancer care clinical indicators classified under six groupings — access, assessment/treatment planning, treatment/support services, outcomes, follow‐up and long term outcomes, and patient‐reported outcome and experience measures — through an iterative process over a 12‐month period (Box 2). The selection was weighted towards ease of collection, data availability and clinical relevance, noting that participation by institutions is voluntary and that they can select the indicators they feel they can most easily complete. Not surprisingly, the final indicator set is predominantly rate‐based, expressing the number of complying or non‐complying events in relation to a given denominator.

The final set of indicators was mapped against the current International Classification of Diseases, tenth revision, Australian modification (ICD‐10‐AM) code set to allow for clinical coding to source data, and was also mapped against the National Safety and Quality Health Service Standards for quality improvement and performance. A comprehensive user manual was prepared, which proceeded smoothly through the endorsement processes, allowing the set to go live for use in July 2020.

There is now wide recognition of the importance of measuring quality in health care provision, with many national bodies around the world having initiated processes to define and quantify quality care goals.18,19,20 Moreover, patients and caregivers are increasingly seeking information on quality over and above information about efficacy, toxicity and cost. The development of clinical indicator sets has been undertaken for many diseases and conditions to facilitate understanding of care provision at a population level. Not only are clinical indicators useful to detect underuse or delay in receiving care but they can also highlight overuse and overtreatment, which has significant short and long term impacts for individual patients and health budgets.21,22 Variation can also be measured across periods of service disruption, exemplified by the coronavirus disease 2019 (COVID‐19) pandemic. When deriving clinical quality indicators, the aim is to make sets as practical and useful as possible for their stated purpose, including processes for regular revision and updating.

With regards to cancer, ACHS members recognised that the development of a high level, nationally applicable assessment of care throughout the cancer journey in Australia was timely. Initially, the task of compiling a compact list of clinical indicators to encompass the cancer care pathway, reached through the consensus of diverse stakeholders, was seen as daunting. With the advice that a maximum of around 20 indicators was ideal, concerns regarding depth and granularity of data and the specificity required for individual tumour populations initially clouded discussions, recognising that there is a vast literature of clinical indicators specific to various tumour types, stages and treatments.13,23 However, the practical considerations of compiling a useful, overarching set of cancer indicators that would be voluntarily selected by institutions seeking accreditation led to discussions being reframed away from specifics and onto a very generic level. This then simplified the task of combining evidence with expert consensus, through the iterative modified Delphi process. The burden of indicator choice was lessened by the design of the ACHS process, which incorporates future reviews to cull indicators that remain unused and with the opportunity to substitute in revised sets.

In the present process, the main reason for rejecting indicators was the concern that data collection would be too onerous for the organisation. This was either because the information would likely need to be aggregated from multiple and disparate sources or may not have been recorded at all. Despite the recognition that the burgeoning amount of digital data should be easy to harness to inform patient‐valued care, quality initiatives, and policy guidelines, the systems of electronic recording of health processes remain basic in many Australian settings.24 Preference was therefore given to indicators sourced from data that are widely collected as routine for health facility systems (bookings, rudimentary electronic medical records, financial systems etc). On the other hand, it is recognised that requiring information for clinical indicators can drive organisations to consider adding or redesigning data collection to facilitate compliance.

A key strength of the process was the undertaking of two full day, face‐to‐face meetings. Processes relying solely on online participation are more subject to levels of fluctuating engagement and involve less direct interaction between participants.²⁵ The in‐person meetings allowed for robust debate as well as finessing of final wording to reduce ambiguity, as terminology often has subtly different usages between craft groups.

In summary, this is the first iteration of a high level, comprehensive clinical indicator set to measure care across the cancer journey of any patient in Australia, from diagnosis to treatment and survivorship. The expert group, consensus‐based methodology with broad stakeholder representation should ensure that this set is easy to use and productive in attaining baseline and comparative quality data to monitor, evaluate and benchmark progress in the provision of cancer care. The set will be regularly reviewed with the potential to add or change clinical indicators, both in response to the experience of the reporting organisations and to capture the changing landscapes of cancer care provision.