Using what we gather - information for improved care

Currently available data can be used to focus clinical quality, patient centredness and safety of care in hospitals Australia has traditionally focused its public reporting efforts concerning hospital care on indicators of volume, costs, length of stay and efficiency at state, territory and national levels. Curiously, there is much less nationally consistent hospital-level reporting on other dimensions of care such as appropriateness, effectiveness, patient-centredness and safety across Australia. There are two major reasons for measuring, monitoring and reporting on these dimensions of quality of hospital care. First, hospital-level reporting stimulates and focuses quality improvement initiatives that support better care and better health. Quality improvement techniques such as benchmarking, Six Sigma, “lean” programs, collaboratives and process re-engineering all depend on measurement and reporting to monitor impact. There is evidence, for example, that confidential, hospital- and physician-level reporting can substantially reduce 30-day mortality after cardiac surgery.1 A review of international evidence indicates that there is “strong and consistent evidence that public reporting stimulates quality improvement in hospitals” and “the majority of studies show significant positive impact of public reporting on clinical outcomes”.2 Second, hospital-level reporting is necessary for accountability and transparency as governments, insurers and the public reasonably expect to understand how effectively care is being delivered. Public reporting is also necessary for transparency if ready access to information is expected to influence patient choice. A recent review of international evidence indicated that public reporting “may be able to make significant and policy-important changes in consumers’ decisions in choosing hospitals in some settings”.1 This Supplement features articles that describe how currently available data can be used to focus quality improvement and to support accountability and transparency through the creation and use of timely and accurate information on clinical quality, patient centredness and safety of care in hospitals. The article by Sketcher-Baker and colleagues on the use of variable life-adjusted displays (VLADs) describes how inpatient data in Queensland have been used to support quality improvement, accountability and transparency.3 Data collation, calculation of VLADs, and feedback inform a clinical improvement program and support accountability while delivering transparency through public reporting of outcomes. Other important elements of this program are the commitment to ongoing review and consultation around the indicators, and the clinical governance model that underpins the VLAD review and response cycle. The article by Clarke and colleagues on the AusPSI program describes how inpatient data have been used to support routine reporting to the Patient Safety Monitoring Initiative in Victoria.4 This initiative involves the use of risk-adjusted outcome measures that build on patient safety indicator work established by the Agency for Healthcare Research and Quality.5 The main questions to ask in assessing data collections that report on quality of care include: Are we asking the right questions — will each data collection accurately describe significant variance in practice and outcomes? Is the data collection feasible and efficient, or unrealistically burdensome? Is there high-quality data scrutiny — of accuracy and reliability? Is there high-quality interpretation and clinical review of reported compliance and variance? Is the risk adjustment fair? Reid and colleagues describe the Australian Cardiac Procedures Registry (ACPR).6 The ACPR includes patient, procedure and outcome data from 21 participating facilities, generating and feeding back risk-adjusted outcome measures against local and international benchmarks. In a previous issue of the Journal, McNeil and colleagues recommended the establishment of clinical quality registries for high-cost, high-volume interventions where there is variation in practice and where practice modification can improve outcomes.7 The successes of the National Joint Replacement Registry,8 the National Breast Cancer Audit,9 the Australian and New Zealand Intensive Care Society Centre for Outcome and Resource Evaluation patient databases,10 and the Australia and New Zealand Dialysis and Transplant Registry11 suggest that clinicians are prepared to trade off the burden of submitting a succinct dataset — that they themselves have developed — in return for routine reports showing their own performance, risk-adjusted, against their peers. Ben-Tovim and colleagues describe their efforts to measure standardised, in-hospital death rates.12 The authors, in collaboration with the Australian Institute for Health and Welfare (AIHW), have led national analyses of hospital mortality data, and refined the Canadian risk adjustment through detailed analyses of the National Hospital Morbidity Database. The calculation, monitoring and reporting of hospital-standardised mortality ratios (HSMRs) is not without controversy.13,14 The Australian Commission on Safety and Quality in Health Care, however, has recommended that hospitals routinely review HSMRs, deaths in low-mortality diagnosis-related groups and condition-specific inhospital mortality rates to identify opportunities to improve hospital care.15 Kennedy and colleagues describe the importance of clinical practice guidelines, and the need to identify and respond to variations in practice, in the context of the rapid escalation of the comparative effectiveness agenda in the United States.16 Leathley and colleagues summarise the results of a forum, convened by the Clinical Excellence Commission in New South Wales, on measuring hospital performance.17 The authors outline key principles for the design of hospital performance measures, and identify measures with the highest potential. McNeil and colleagues describe the National Antimicrobial Utilisation Surveillance Program approach to measure, monitor and identify significant variance and trends in antibiotic usage.18 They demonstrate how antibiotic usage data from 28 principal referral hospitals and one private hospital have generated interventions and real change in antibiotic prescribing practice. Current health care reportingAt the national level, the AIHW and the Australian Government Productivity Commission take seriously their charters to “provide information on Australia’s health and welfare, through statistics and data development”19 and “promote public understanding of matters related to industry and productivity”20 on health services. On 20 April 2010, the Council of Australian Governments agreed (with the exception of Western Australia) to sign the National Health and Hospitals Network Agreement, including the establishment of a National Performance Authority, and there are plans to launch a public website with hospital-level information.21 At the state and territory level, several governments release information on the performance of public hospitals, including Victoria,22 Queensland23 and NSW.24 NSW publishes information including waiting lists for elective surgery, health care-associated infections and current safety notices, and hosts a health service website.25 The NSW Bureau of Health Information was recently established to publicly report on the performance of the state’s health system. The Bureau’s first report provided comparative, hospital-level information on patient-centred care in 38 large hospitals.26 Its hospital quarterly reports will provide information on inpatient services, surgical care and emergency departments every 3 months. The first issue expanded the scope of hospital-level information previously reported to include new measures of accessibility and patient-centred care and increased the number of hospital emergency departments reported on from 40 to 66.27 The way forwardIf we accept the premise that timely, accurate and comparable information about the performance of hospitals is “a good thing”, the most important question remains: “What measures are meaningful and useful?” If the purpose of reporting is better care, selection of measures should be driven by the priorities of clinicians and health care management and policy communities. If the purpose of reporting is accountability, selection of measures depends on the aim of investments. If the purpose is transparency about the performance of hospitals, a broad and balanced portfolio of measures is important. Authors of articles in this Supplement highlight how current information systems in Australia can be used to gather meaningful, useful information for clinical, management and policy communities. It’s up to the rest of us to build on these initiatives to create and use timely, accurate and comparable information about the performance of hospitals in Australia to support better care.

Neville Board RN, BA, MPH · Diane E Watson PhD, MBA, BScOT

A Clinical Excellence Commission seminar explored how clinical practice variation can be monitored, and identified directions and opportunities in this field The Clinical Excellence Commission (CEC) is a statutory corporation, entrusted with improving patient safety and clinical quality within the New South Wales health system, in collaboration with key partners. In addition to sponsoring several clinical improvement initiatives, the CEC has a reporting, capacity-building and networking role to identify, promote and help spread best practice. In 2008, to meet community and government requirements of transparency, openness, accountability and integrity, the CEC sought to progress the development of a small set of safety and quality measures for NSW hospitals. Related reform initiatives included the Special Commission of Inquiry into Acute Care Services in NSW Public Hospitals1 and outcomes of the 2008 Council of Australian Governments meeting,2 calling for the development of robust, agreed, reliable, readily available and publicly reported measures relating to quality and safety of health care. Cognisant of successful developments overseas and nationally, and of reform initiatives within NSW, the CEC hosted a seminar in Sydney on 13 November 2008, entitled Hospital Performance and How We Can Measure and Report On It. The aim of the seminar was to explore current reporting mechanisms and to engage a range of key stakeholders in discussion regarding the development of critical measures that could best report quality and safety performance in NSW hospitals. Over 50 senior clinical and administrative stakeholders were in attendance. The seminar focused on the experiences of health agencies in Canada and Australia, identifying initiatives for capturing and reducing variation, and highlighting challenges and opportunities related to data collection and reporting. Outcomes from the seminar included identification by the assembled group of agreed key principles, critical measures with the highest potential and the next steps to facilitate progress. PresentationsThe seminar included the following presentations: The Canadian Experience of Performance Measurement and Reporting (Professor G Ross Baker, University of Toronto) shared the experience of developing a balanced scorecard in the Canadian health system in the mid 1990s and of the growing acceptance of public reporting of key indicators. Measures for the Australian Health System (Jenny Hargreaves, Australian Institute of Health and Welfare) outlined a set of proposed performance indicators across the health and aged care system, and the development of national indicators of safety and quality in health care, being undertaken for the Australian Commission on Safety and Quality in Health Care. The Queensland Experience (Professor Michael Ward, Health Quality and Complaints Commission) described the origins and evolution of public hospital performance reporting via variable life-adjusted displays (VLADs). The Overview (Professor Clifford Hughes, CEC) outlined key drivers and developments of developing public hospital performance measures in the NSW health system. DiscussionA plenary discussion after the presentations debated the need for improved measures of safety and quality, the requirement to report these publicly, and the potential role of NSW Health or the CEC in measuring hospital performance. The following key points were identified during the discussion. Improvement and accountability are interlinked. There is value in distinguishing between dimensions of care; for example, low-dimension elements with clear intervention, outcomes and accountabilities, and high-dimension elements, which are more complex and where outcomes, processes and accountabilities are less clear. Community expectations and patient satisfaction measures do not generally correlate with performance outcome measures. There is a need to consider timing correlations between performance reports, reduction in variation and improved quality of care. VLAD data provide immediate feedback with links to process changes relatively evident, whereas mortality-related data take longer to review. Triangulation approaches are likely to assist. Linkage to population health is supported, with development of relevant measures being considered. “Accountability” generally involves elements of blame and responsibility for improvement. A key question to address is: “Accountability to whom?” Accreditation is a useful part of the framework, but is not an end in itself. Mortality is a complex outcome, not simply a “bad” one, and its negative aspects need to be balanced against allowing a patient to die with dignity and respect. Datasets will need to be refined over time, in terms of number and value, rather than trying to get a perfect set first time. The value for hospitals and clinicians is in being able to compare performance with their peers in regard to valid measures. Feedback loops are important for sharing and acting on data — there is a need to link in with broader quality improvement processes and other parts of the system, such as ambulance services. Systematic, regular reporting is seen to be important for the health system (from Minister to clinicians) and consumers, to provide reassurance that the system is performing as it should, and to highlight vulnerabilities to be improved. Quality of data is more important than quantity of data. There was general consensus that the key principles shown in Box 1 should apply to the design of a set of hospital performance measures for safety and quality. Selecting measures with greatest potentialSeven working groups were asked to identify the indicators or measures with the highest potential, in terms of being: relevant; clearly defined; measurable; routinely reportable; robust (high-volume, reliable, clearly defined); evidence-based and/or representative of interventions that will most improve safety and quality; timely; risk-adjusted; consumer- or patient-focused; immune to political influence, “gaming” or manipulation, and perverse incentives; and able to be collected with minimal cost and burden on clinicians. The measures with the highest potential as selected by all groups are shown in Box 2. The group believed that this would be an ambitious, but realistic, initial set of measures, to be refined or developed with experience. Other measures which were considered to have potential, but not to meet all criteria at this time, included: caesarean section and other women’s health intervention rates including hysterectomy and episiotomy; stroke and heart failure best-practice care (bundle of evidence-based interventions as per acute coronary syndromes); mental health (readmissions, number of admissions per annum, follow-up after 7 days); hospital-acquired malnutrition; mortality from conditions considered amenable to health care; hospital standardised mortality rates; staff satisfaction; and open disclosure process. Subsequent directionsSeminar participants endorsed the CEC proceeding in partnership with other key stakeholders in developing and implementing a key set of indicators for reporting safety and quality in NSW hospitals. As part of this process, a summary report of the seminar was distributed to participants and temporarily posted on the CEC website to communicate, lead discussion and increase buy-in. This included acknowledgement of the need to engage more broadly with significant groups within hospitals in the development, collection and reporting of relevant measures. Shortly after the seminar, the final report of the Special Commission of Inquiry into Acute Care Services in NSW Public Hospitals was published.1 A key recommendation of the report was identification, development and publication of patient care measurements as a comprehensive way of seeing how patients in NSW hospitals are being looked after. Key measurements identified by the report were: access to and availability of hospital services; clinical performance; safety and quality of clinical care and hospital attendance and admission; cost of clinical care; patient experience and satisfaction; staff experience and satisfaction; and system impact and sustainability. The report recommended the establishment of a Bureau of Health Information to meet the above reporting requirements. This Bureau (http://www.bhi.nsw.gov.au/) was formally established in 2009, and the CEC will work in close association with the Bureau to support and help promote the development, implementation and reporting of relevant quality and safety measures. The findings of the seminar reported here will be a key factor in this development. 1 Key principles Choose measures based on strategy and intent, not political imperative Identify the core purpose (eg, accountability, improvement, research, consumer/patient knowledge) Choose a limited number of measures Have dual sets of indicators for different purposes: high-level indicators for public reporting more detailed outcome and process measures for quality improvement Engage clinicians in the design and collection of indicators; this is crucial Increase capacity of computerised systems to facilitate access to outcome measures, and to reduce the burden of data collection Recognise that most improvements in indicator data quality usually follow their reporting 2 Selected measures Hospital-acquired infections (bundle of measures including methicillin-resistant Staphylococcus aureus infection, infection with vancomycin-resistant enterococci, central line infections, surgical site infections, Clostridium difficile infection, and ventilator-associated pneumonia) Pressure ulcers Best-practice care for acute coronary syndromes (bundle of evidence-based interventions including provision of medications on discharge) Unplanned return to intensive care unit Unplanned return to operating theatre Medication errors (with associated measures of extent of harm) Patient falls Management of patients with deteriorating conditions Venous thromboembolism 30-day unplanned overnight readmission rate

Colleen M Leathley MSocSc(Hons) · Richard Gilbert BSc(Hons) · Peter J Kennedy MB BS, FRACP · Clifford F Hughes

Although difficult to quantify, there is known widespread variation in the way that best available evidence is applied in clinical practice. The reasons for gaps between evidence and practice are complex, and efforts to improve uptake are unlikely to be successful if they are one-dimensional or focus on individual health professionals. This article provides contextual reference for articles in this Supplement in addressing how and why clinical variation exists, the importance of reducing it and strategies to drive a more streamlined approach to evidence-based care in Australian health care systems.

Peter J Kennedy MB BS, FRACP · Colleen M Leathley MSocSc(Hons) · Clifford F Hughes MB BS, FRACS, FACS

Worldwide, current practice is to report hospital mortality using the hospital standardised mortality ratio (HSMR). An HSMR is generated by comparing an indirectly standardised expected mortality rate against a hospital’s observed mortality rate. A hospital’s HSMR can be compared with the overall outcomes for all hospitals in a population, or with peer hospitals. HSMRs should be used as screening tools that alert institutions to the need for further investigation, rather than as definitive measures of the quality of care provided by individual hospitals. HSMRs are computed from existing hospital administrative data sources, which are fit for such a purpose. The addition of clinical or physiological data does not, at present, add to the discriminative powers of the risk adjustment models used to adjust HSMR values for differences in hospitals’ casemixes. There has been concern that HSMRs may be too variable over time for individual values to be interpretable. A study of HSMR outcomes in Australian hospitals confirmed earlier reports of the stability of the measure. Considerable progress has been made with developing Australian HSMRs for use as routine measures to improve the safety and quality of Australian hospital care.

David I Ben-Tovim PhD, MB BS, FRANZCP · Sophie C Pointer BSc(Hons), PhD · Richard Woodman BSc(Hons), PhD, MBiostat · Paul H Hakendorf BSc, MPH · James E Harrison MB BS, MPH

The variable life-adjusted display is a graphical, statistical methodology used in Queensland to monitor patient outcomes of clinical indicators. The quality improvement cycle is a systematic approach employed by patient safety and quality programs worldwide to improve patient care. The quality improvement cycle is beneficial to the review and refinement of indicator definitions. Indicators with definitional issues that are not subject to the quality improvement cycle may initially prompt quality improvement opportunities, but are more likely to potentially lead to unnecessary chart and clinical reviews, which will disengage coders and clinicians. Queensland recently used the quality improvement cycle to refine the laparoscopic cholecystectomy complications of surgery indicator definition and several maternity definitions.

Kirstine M Sketcher-Baker BAppSc(Maths) · Maarten C Kamp MB BS, MHA · Julia A Connors BNurs, BAppSci(MedSci), BAppSci(Hons) · Don J Martin MB BS, BBus · Justin E Collins GradCertHlthMgt

Routine monitoring of performance in the provision of cardiac services aids quality assurance and enables comparisons of performance to national and international standards. The Australasian Society of Cardiac and Thoracic Surgeons conducts a surgical registry that has grown from six hospitals participating in 2001 to 21 contributing in 2010. Variation in performance is monitored on a quarterly basis through the use of control chart methodology, and a peer-review mechanism and governance process for reporting have been established. Proposed future developments of the registry include its expansion to include interventional cardiology procedures, such as implantation of stents and cardiac devices, and a modular format, with the patient rather than the procedure being the key element of the system. An Australian Cardiac Procedures Registry will provide information to stakeholders, including consumers, clinicians, health funders and policymakers, on performance standards and quality of care of medical services affecting an ever-increasing number of Australians.

Christopher M Reid DipEd, MSc, PhD · Angela L Brennan RN, CCRN · Diem T Dinh BSc(Hons), PhD · Baki Billah PhD, MAS, MSc · Carl B Costolloe BSc(Photonics), MEng · Gilbert C Shardey MD, FRACS · Andrew E Ajani MB BS, MD

This article reports the experience of the Victorian Department of Health in seeking clinician engagement in the testing of 11 quality-of-care indicators in 20 health services in Victoria. The Department previously developed a suite of 18 core indicators and seven subindicators known as the AusPSI set. We used routinely collected administrative data from the Victorian Admitted Episodes Dataset to produce variable life-adjusted display (VLAD) control charts for 11 selected indicators. The Department recognises that clinicians are responsible for the safety and quality of the care they provide, and therefore the necessity of engaging clinicians in the process of investigating apparent variation in patient care. Although using readily available and inexpensive routinely collected administrative data to measure clinical performance has a certain appeal, the use of administrative data and VLADs to identify apparent variations has posed significant challenges due to concerns about the quality of the data and resource requirements. When clinicians at a major Melbourne hospital were engaged, it resulted in an improvement in clinical practice. Investigating apparent variation in patient care provides an ideal opportunity for emerging clinical leaders to take local ownership and develop expertise in investigating apparent variation in processes of care and implementing change as required.

Andrew L L Clarke RN, BAppSci(Nursing) · William Shearer MB BS, MEdSt, FANZCA · Alison J McMillan RN, BEd, MBA · Paul D Ireland PhD, MSc, BSc(Hons)

The National Antimicrobial Utilisation Surveillance Program (NAUSP) collects aggregate data from hospitals in all Australian states and provides reports of monthly hospital inpatient antimicrobial usage to contributing hospitals. These data provide an Australian peer-group benchmark; hospitals can compare their usage with similar hospitals and identify areas of antimicrobial use that require more indepth analysis. Overall high usage has been used by hospitals and area health services as a stimulus for initiation or expansion of antimicrobial stewardship programs. High use of particular classes of antimicrobials has triggered individual drug audits and been used to tailor interventions. Longitudinal antimicrobial usage data have been used by hospitals to measure the effects of antimicrobial stewardship strategies and provide feedback to prescribers.

Vicki McNeil BPharm, GradDipPharm, GradCertPubHlth · Marilyn Cruickshank RN, PhD · Margaret Duguid BPharm, GradDipAdmin