The recent controversy in Victoria over the manipulation of waiting lists in a number of public hospitals has focused attention again on the problems associated with a performance management system that is not underpinned by the collection and reporting of sound data.

An independent audit of Melbourne’s Royal Women’s Hospital found that patients awaiting urgent or semi-urgent elective surgery whose waiting times were approaching the target for their category (30 days and 90 days, respectively [Box]) were reclassified as “not ready for care — patient initiated”. The reclassification ensured that category waiting time targets were not exceeded and that the hospital met elective surgery key performance indicators.1 Although at the time of the audit the hospital did not participate in the bonus scheme designed to reward hospitals for achieving targets, the state Minister for Health has since scrapped the bonus scheme.2

During the ensuing public debate, the Victorian Auditor-General and the Australian Institute of Health and Welfare (AIHW) both released reports highlighting the limitations of waiting times and clinical urgency categories as indicators of patient access to elective surgery and hospital performance.3,4

Taken at face value, waiting times for elective surgery appear to be a simple measure that can provide information about patient access to services, as well as hospital performance. However, the measurement of waiting times and interpretation of waiting times data are inherently complex. Because we currently measure only the period from the date of entry to the waiting list until provision of surgery, improvements in waiting times do not reflect the entire patient journey and may be negated by increases in the time spent waiting for primary care, diagnostic testing or specialist outpatient appointments. And although waiting times may tell us how long it takes for patients to move through one part of the care pathway, they tell us nothing about the quality of care received. Additionally, their use as performance indicators may provide a motive for data manipulation.5

The clinical urgency categories that have been used as part of the waiting-list management process also have important limitations. Patients are classified into three broad urgency categories (Box) by a largely subjective process with poor reproducibility between clinicians and across health services.6 While the intensity of clinical symptoms such as pain has been fundamental to assigning urgency categories, clinicians vary in how they assess these symptoms. In assigning urgency, clinicians may also consider non-clinical factors, but this occurs informally. Further, each urgency category is aligned with recommended waiting times that are not evidence-based.3 In recognition of the problems with jurisdictional variability, the AIHW has excluded urgency categories from its national reporting since 1999.

These limitations make the current elective surgery categorisation system unsuited for one of its main purposes — stimulating improvement through benchmarking and the use of performance targets. Lack of reproducibility and vulnerability to manipulation, which are characteristics of some of the system’s access and performance indicators, limit the value of the information collected. This is one of an increasing number of instances where inadequate attention to data quality has compromised the important function of providing accurate information to guide service provision.

If waiting times are to be a sound basis for the routine reporting of patient access to elective surgery or other health services, then they should be explicitly defined to provide a valid assessment of the time spent waiting for surgery, and measured in a standardised and unambiguous way. This approach has been advocated in Canada, where there has been a concerted attempt over the past 5 years to establish evidence-based waiting time benchmarks for specific procedures.7 Likewise, the categorisation of urgency needs to be underpinned by an explicit and standardised way of taking into account all factors relevant to a patient’s requirement for elective surgery. This has already occurred to varying extents in New South Wales, New Zealand and Canada. In NSW, recommendations for assigning specific surgical procedures to urgency categories have been developed and set in policy.8 NZ and Canada have both developed explicit, practical tools that help clinicians systematically prioritise patients for a range of specific procedures, such as joint replacement and cataract surgery.9,10 A clinical priority access system, while not without its controversies, has been used throughout NZ since the early 1990s. Some of the prioritisation tools developed by the Western Canada Waiting List Project have been used in elective surgery programs in several Canadian provinces.

The challenges in finding reliable measures of health system performance are not unique to waiting lists. In every area of the health care system, problems result when data used to assess access, quality of care and safety are not based on appropriate measurements. The cornerstones of epidemiologically sound data are validity and reproducibility. The identification of measures able to produce valid, reproducible data is essential for credible assessment of health system performance and to drive long-term improvements.