Connect
MJA

Measuring safety and quality to improve clinical outcomes — current activities and future directions for the Australian Cardiac Procedures Registry

Christopher M Reid, Angela L Brennan, Diem T Dinh, Baki Billah, Carl B Costolloe, Gilbert C Shardey and Andrew E Ajani
Med J Aust 2010; 193 (8): S107. || doi: 10.5694/j.1326-5377.2010.tb04024.x
Published online: 18 October 2010

Cardiac interventional procedures — including cardiac surgery, percutaneous transluminal coronary angioplasty (PTCA) and implantation of cardioverter-defibrillator devices — have been major therapeutic advances in cardiovascular medicine and have led to declining coronary heart disease case fatality rates over the past few decades.1 These therapeutic advances are affecting health services around the world;2,3 the Health, United States, 2009 report included a special section on advances in medical technology.3 The report stated that the use of medical technology has tripled in the past decade and of the top five costliest procedures, three are cardiovascular and include PTCA (a 65% increase in use with a 108% inflation-adjusted cost increase between 1999 and 2006), implantation of medical devices including cardiac pacemakers and defibrillators (up 147%), and coronary artery bypass grafting (CABG) procedures (down 24%).3

There has also been a steady increase in the volume of these procedures conducted around Australia. In 2004, the Australian Institute of Health and Welfare estimated that cardiac surgical procedures were undertaken in over 50 public and private centres in Australia.1 CABG was the most common cardiac surgical procedure, followed by replacement of the mitral and aortic valve.1

The number of PTCA procedures performed in Australian hospitals has grown steadily, and with the introduction of new technologies (particularly drug-eluting stents) the cost per procedure has also escalated. In 2006–07, the Australian Institute of Health and Welfare reported over 35 000 PTCA procedures with or without insertion of bare-metal or drug-eluting stents.4 This rapidly evolving technique has been shown to be effective at relieving symptomatic angina and improving prognosis in patients with acute coronary syndromes.

Among people aged over 75 years, there was a fourfold increase in procedures over the same period, indicating a preference by older patients and/or their doctors for percutaneous coronary intervention (PCI). Since 1997–1998, PCI has replaced CABG as the most common coronary revascularisation treatment for coronary heart disease in Australia.1 Over the past few years, there has been a dramatic increase in PCI procedures (based on hospital separations in which a PCI was performed) in Victorian public hospitals, from 3644 in the 2003–04 financial year to 4561 in 2004–05, representing a 25% increase in procedures (Daniel Borovnicar, Victorian Government Department of Human Services, personal communication, April 2006). There was a 43% increase in the total cost over the 2 years. This increase is likely driven by the establishment of new catheterisation laboratories with PCI capability, but is also due to the higher cost of drug-eluting stents, which were introduced during this period.

Furthermore, the increasing use of implantable devices has created a range of imperatives similar to those posed by therapeutic drugs. An estimated 4% of the US population has at least one implanted medical device. A series of unanticipated failures was identified after widespread clinical use, resulting in urgent and costly recalls.5 The surveillance systems for devices such as pacemakers and implantable cardioverter-defibrillators increasingly involve registries with regular patient follow-up.

In the US, the Society of Thoracic Surgeons6 and the American College of Cardiology7 have established registries with the aim of improving the quality of care and clinical outcomes for surgical and interventional procedures, respectively. Similar initiatives have been undertaken in the United Kingdom8 and Europe;9 these have led to important developments including models for risk adjustment, enabling comparisons of outcomes on a risk-adjusted basis. The necessity for registries for quality performance and safety monitoring was demonstrated following concerns about the quality of individual surgeon performance in cardiac surgery in the Bristol inquiry.10 The inquiry highlighted the need for performance benchmarking and quality assurance in high-risk procedures, such as cardiac surgery, and sparked a dramatic growth in this area.8 Similar calls for the establishment of cardiac procedural registries in Australia have been made.11,12

At around this time, the Australasian Society of Cardiac and Thoracic Surgeons (ASCTS) recognised the need for a similar system for performance monitoring to be developed and trialled in Australia; this gave rise to the ASCTS Victorian cardiac surgery database project being initiated in 1999.13 The purpose of the registry was to establish a systematic approach to cardiac surgery data collection and reporting. This enables benchmarking of performance standards across hospitals to assure quality performance and improve clinical outcomes for patients undergoing cardiac surgery procedures.

Here, we describe how the ASCTS surgical registry routinely collects and compares data to assure safety and quality and drive improvements in clinical outcomes. In addition, we discuss future directions for quality and safety initiatives.

Methods

Financial support for the establishment and conduct of the ASCTS Victorian Database Project was provided by the then Victorian Department of Health and Aged Care. A steering committee was established to establish governance and procedural activities for the establishment of the registry. To appropriately undertake comparisons of variation in performance, it was recognised that a standard definition set, clearly defined performance indicators, appropriate analysis and risk adjustment, and a peer-review monitoring and audit system were fundamental requirements.

Establishing a data definitions set

Despite initial funding for the activity in six Victorian public hospitals, it was recognised that national engagement was required for an agreed set of variables. The ASCTS established a National Dataset Committee, which reviewed existing available cardiac surgery datasets and definitions to establish a standard set of data elements for collection in Australia. It was agreed that the dataset would remain stable and be revisited every 2 years for additions and changes according to feedback. The initial dataset comprised approximately 200 variables, separated into Patient Demographics, Risk Factors, Pre-operative Cardiac Status, Previous Cardiac Interventions, Haemodynamic Data, Operation Status/Category, Procedural Details, Postoperative data and Mortality, Discharge and Readmission data.14

Establishing performance indicators

As an integral part of the funding agreement, a set of key performance indicators was identified. A mix of clinical and process indicators was agreed upon and comprised:

  • 30-day mortality after isolated CABG;

  • return to operating theatre for deep sternal wound infection;

  • return to operating theatre for haemorrhage;

  • ventilation time;

  • length of stay in intensive care; and

  • total length of hospital stay.

Data collection process

Standardised forms were developed for collection of ASCTS data, and a desktop application based on Microsoft Access was provided to each participating site. Staff at the sites were trained in data form completion and application use, and a protocol for submission of data to the central data repository was established. Central data queries were generated and sent for review at the local sites. Sites were required to submit data quarterly for the review. The steering committee met quarterly to review data submitted for the previous period.

Data analysis and handling of outliers

The identification of performance outside an agreed standard was made on the basis of statistical process control charts. In recent years, additional techniques including cumulative sum (CUSUM) control charts and variable life-adjusted displays have been included in the detection of outliers and variation of outcomes between units. The committee recognised the requirement for appropriate risk adjustment and the EuroSCORE (European System for Cardiac Operative Risk Evaluation) method was chosen.9 A major objective of the project was for the validation of the EuroSCORE for risk adjustment in the Australian context and the development of an Australian risk model if required.

A peer-review mechanism process for the registry involves notification to sites of any outlier identified in a quarter. This allows for data reported in the outlier period to be checked and confirmed prior to the next review cycle. This activity is conducted as an internal communications process between the registry and the site. Sites that fall outside of agreed standards on any performance indicator for two consecutive periods are asked to participate in a review process with the Peer Review Committee of the ASCTS. Annual reporting of the cardiac surgical registry activity was proposed on three levels:

  • a public report, outlining surgical activity and performance indicators to be made available through the state government web-site for community access;

  • a hospital-specific report, identifying the individual hospital and its performance against other participating units; and

  • a comprehensive surgeons’ report, sent to all ASCTS members, providing data on surgical activity in Victoria.

Data quality assurance

Onsite data audits were conducted on a random selection of sites to be monitored at least once every 3 years. At monitoring visits, a random selection of cases was reviewed for completeness and verification of key data submitted to the registry.

Results

Commencing in 2001, with six Victorian public hospital units and collecting data on 2791 procedures, the ASCTS database registry has grown to contain over 30 000 surgical procedures. In 2010, the performance of 21 public and private cardiac surgical units around Australia will be monitored, and participation is increasing. Box 1 illustrates the 30-day mortality and deep sternal wound infection performance indicator reporting for 18 participating hospitals for the 2008–09 financial year using control charts. Box 2 illustrates unit performance against the mean performance from the participating centres and identifies UK and US reported mean mortality rates for comparison. Similar methods for monitoring are used for each of the five performance indicators for cardiac surgery on a quarterly basis.

At this stage, an Australian risk adjustment model for 30-day mortality has been developed, and future research will focus on appropriate risk adjustment for other performance indicators, including return to theatre for sternal wound infection and haemorrhage.15

Discussion

Over the past 10 years, Victorian public hospital outcomes for cardiac surgery have been closely monitored, and at no stage has any unit been identified as an outlier requiring external peer review. In recent years, we have explored a variety of alternative methods for reporting and monitoring performance of participating units, including variable adjusted life displays, CUSUM methods and funnel plots. Communicating and expressing variation in performance is a major challenge as simplistic approaches such as ranking (league tables) fail to identify a lack of variation between centres. In an analysis of 175 ranked cardiac surgeons in New York state, the confidence intervals around risk-adjusted mortality were so wide that only two could be considered in the lowest mortality quartile, and only six could be considered in the highest mortality quartile.16 The perception of a difference due to ranking in this instance is completely misleading.

In 2009, the Australian Commission for Safety and Quality in Health Care called for tenders to test and validate the operating principles and technical standards for clinical quality registries. ASCTS, in collaboration with the Melbourne Interventional Group, proposed the development of the web-based Australian Cardiac Procedures Registry (ACPR) to monitor cardiac procedural activity in surgery, PCI and device implantation.17 The concept of a modular framework capturing the “cardiac patient journey” across a variety of interventions has been supported by the Cardiac Society of Australia and New Zealand. In the first year, it has focused on establishing standardised datasets and performance indicators for each of the three areas of clinical activity (Box 3). This will enable further evaluation of the impact of care on individuals rather than the focus on procedures inherent in separate procedural registries.

Future developments to the ACPR incorporating online and real-time performance monitoring for individual practitioners are identified as further enhancements to the system. Issues of privacy, in particular user authentication and access rights for web-based systems, are identified as key areas for regulation and governance.

The target population for ACPR activity is patients, practitioners and policymakers. A cardiac procedural registry is crucial to ensure that all patients are getting the best possible treatment and achieving the best outcomes. Patients are entitled to know the likely outcome of various treatments, with appropriate risk adjustments, in order to make informed choices. A cardiac procedures registry would be an invaluable tool for clinicians, allowing them to review practice, compare outcomes to a “standard” and make changes to improve practice. As evidenced by increased hospital participation with the ASCTS Registry, surgeons can see the value in participating in a quality assurance program where their unit and their own individual performances have been benchmarked. Comprehensive annual reports have been published along with peer-reviewed manuscripts that highlight the overall success of the registry. Individual sites performing coronary revascularisation, and implanting cardiac devices procedures have only internationally published cohort data with which to compare outcomes. Using the framework established through the ACPR establishment, procedural and outcome data will enable Australian proceduralists’ performance to be quantified, internally and against international benchmarks.

Policymakers are entitled to know how well clinical cardiac services are provided to the community in order to make better choices, identify centres of excellence and develop strategies for quality improvement. The ACPR will make this information available to the target population while protecting the privacy of individual patients and clinicians.

1 Monitoring unit performance for cardiac surgery — 30-day mortality and deep sternal wound infection, 2008–09

A. Control chart — observed 30-day mortality

B. Control chart — deep sternal wound infection


The term fraction defective refers to the variance from the mean for each cardiac unit. Control limits are added to the plot to signal 3 standard deviations from the group mean. In both cases, zero units are out of control.

2 Mortality rate within 30 days following isolated coronary artery bypass grafting for the 18 hospital cardiac surgery units, 2008–09


RAMR = risk-adjusted mortality rate using the AusSCORE model.15

3 Proposed key performance indicators for the Australian Cardiac Procedures Registry

Percutaneous coronary intervention

  • Mortality to 30 days

  • Periprocedural myocardial infarction

  • Target vessel failure to 30 days

  • Target vessel revascularisation

  • Stent thrombosis

  • Urgent inhospital coronary artery bypass grafting

  • Procedural success

  • Major bleeding inhospital

  • Stroke to 30 days

  • Door-to-balloon time

Surgery

  • Mortality to 30 days

  • Deep sternal wound infection to 30 days

  • Unplanned return to operating theatre for bleeding (inhospital)

  • Stroke to 30 days

  • Readmission to hospital

Devices

  • Mortality to 30 days

  • Reoperation

  • Readmission to hospital

  • Procedural adverse events

  • Early procedural complications (< 1 month)

  • Late procedural complications (> 1 month)

  • Christopher M Reid1
  • Angela L Brennan1
  • Diem T Dinh1
  • Baki Billah1
  • Carl B Costolloe1
  • Gilbert C Shardey2
  • Andrew E Ajani3

  • 1 Monash Centre of Cardiovascular Research & Education (CCRE) in Therapeutics, Department of Epidemiology and Preventive Medicine, School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC.
  • 2 Cabrini Medical Centre, Melbourne, VIC.
  • 3 Royal Melbourne Hospital, Melbourne, VIC.


Acknowledgements: 

We acknowledge the following individuals for their contribution to the development and establishment of the ACPR:

ACPR Project Management Committee. Andrew Ajani, Barbara Anthony, Leonard Arnolda, Peter Bissaker, Alexander Black, Angela Brennan, Harriet Carruthers, Derek Chew, Carl Costolloe, Brett Dennett, Ron Dick, Diem Dinh, Stephen Duffy, Leeanne Grigg, Henry Krum, Jeff Lefkovits, Andrew MacIsaac, Leo Mahar, Steven McConchie, John McNeil, Harry Mond, Andrew Newcomb, Silvana Marasco, Al Mutazz Diqer, Chris Reid, Gil Shardey and Peter Subramaniam.

ACPR Data Definition Working Groups

Device. John Atherton, Angela Brennan, Karen Carey, Carmine De Pasquale, Diem Dinh, Henry Krum, Jo-Dee Lattimore, John McNeil, Harry Mond, Gerry O’Driscoll, Chris Reid, David Ross and Cameron Singleton.

PCI. Andrew E Ajani, Angela Brennan, David Brieger, Derek Chew, Diem Dinh, Stephen J Duffy, Christopher Hammett, Jeff Lefkovits, Andrew MacIsaac, Yuvi Malaiapan, Jamie Rankin, Chris Reid and Matt Worthly.

Cardiac Surgery. Peter Bissaker, Daniella Brasacchio, Diem Dinh, James Edwards, Clifford Hughes, David Marshman, Julie Mundy, Andrew Newcomb, Mark Newman, Gil Shardey, Peter Subramaniam and Hugh Wolfenden.

ACPR Steering Committee. Chris Reid, James Cameron, Karen Carey, Derek Chew, Niall Johnson, Larry Kelly, Leo Mahar, John McNeil, Harry Mond, John Rankin, Gil Shardey, James Tatoulis and Andrew Tonkin.

ACPR Web Development Team. Carl Costolloe, Ramya Jaganathan, Philip Scotney and Igor Yeykelis.

Competing interests:

The ACPR was funded by the Australian Commission for Safety and Quality in Health Care (tender RFT 018/0809).

  • 1. Australian Institute of Health and Welfare. Heart, stroke and vascular diseases, Australian facts 2004. Canberra: AIHW, Heart Foundation of Australia, 2004. (AIHW Cat. No. CVD 27.)
  • 2. State of the heart in the USA. Lancet 2010; 375: 697.
  • 3. National Center for Health Statistics. Health, United States, 2009: with special feature on medical technology. Hyattsville, Md: NCHS, 2010.
  • 4. Australian Institute of Health and Welfare. Australian hospital statistics 2006–07. Canberra: AIHW, 2008. (AIHW Cat. No. HSE 55.)
  • 5. Maisel WH. Medical device regulation: an introduction for the practicing physician. Ann Intern Med 2004; 140: 296-302.
  • 6. Ferguson TBJ, Dziuban SW, Edwards FH, et al. The STS National Database: current changes and challenges for the new millennium. Ann Thorac Surg 2000; 69: 680-691.
  • 7. Anderson HV, Shaw RE, Brindis RG, et al. A contemporary overview of percutaneous coronary interventions: the American College of Cardiology-National Cardiovascular Data Registry (ACC-NCDR). J Am Coll Cardiol 2002; 39: 1096-1103.
  • 8. Keogh B, Kinsman R; Society of Cardiothoracic Surgeons of Great Britain and Ireland. 1998 National Adult Cardiac Surgical Database report. London: Society of Cardiothoracic Surgeons of Great Britain and Northern Ireland, 1999.
  • 9. Roques F, Nashef SAM, Michel P, et al. Risk factors and outcome in European cardiac surgery: analysis of the EuroSCORE multinational database of 19030 patients. Eur J Cardiothorac Surg 1999; 15: 816-823.
  • 10. Smith R. Regulation of doctors and the Bristol inquiry. BMJ 1998; 317: 1539-1540.
  • 11. Tonkin A. Why Australia needs a cardiac procedures database. Heart Lung Circ 2001; 10 (1 Suppl): S22-S25.
  • 12. Scott I. Why we need a national registry in interventional cardiology. Med J Aust 2008; 189: 223-227. <MJA full text>
  • 13. Reid C, Solterbeck A, Buxton BF, et al. Developing performance indicators for cardiac surgery: a demonstration project in Victoria. Heart Lung Circ 2001; 10 (1 Suppl): S29-S33.
  • 14. Australian Society of Cardiac and Thoracic Surgeons. ASCTS National Cardiac Surgery Database Program data definitions manual. Version 3. 2008. http://www.ccretherapeutics.org.au/research/ascts.html (accessed Aug 2010).
  • 15. Reid C, Billah B, Dinh D, et al. An Australian risk prediction model for 30-day mortality after isolated coronary artery bypass: the AusSCORE. J Thorac Cardiovasc Surg 2009; 138: 904-910.
  • 16. Jacobs JP, Cerfolio RJ, Sade RM. The ethics of transparency: publication of cardiothoracic surgical outcomes in the lay press. Ann Thorac Surg 2009; 87: 679-686.
  • 17. Ajani AE, Szto G, Eccleston D, et al. The foundation and launch of the Melbourne Interventional Group: a collaborative interventional cardiology project. Heart Lung Circ 2006; 15: 44-47.

Author

remove_circle_outline Delete Author
add_circle_outline Add Author

Comment
Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

Online responses are no longer available. Please refer to our instructions for authors page for more information.