Clinical quality registries for clinician-level reporting: strengths and limitations

Ahern and colleagues1 explore the potential benefits and pitfalls of benchmarked reporting in the Australian context. As a binational registry of patients on renal replacement therapy in Australia and New Zealand, the Australia and New Zealand Dialysis and Transplant Registry has been producing and distributing centre-specific performance reports to renal units for over 20 years; these share many of the challenges faced by clinician-level reporting. In the past few years, this has extended to provision of an abridged version of the report on our website, containing unit-specific risk-adjusted outcome data for each dialysis and transplant unit (http://www.anzdata.org.au/v1/hospitalreport.html).

The authors highlight the challenges of low case numbers resulting in statistical models that are underpowered to detect poor performance and require long observation periods that will limit timely detection of outliers and opportunities for remedial action. Co-opting statistical techniques used for quality control in other industries may present an opportunity to address these issues in the health care sector. Cumulative sum control charts2 provide a method for sequentially monitoring cumulative performance over time, which may permit early detection of poor performance and account for varying activity levels by including the number of procedures performed, rather than just a fixed time frame. Similarly, Bayesian approaches that involve updating prior probability distributions within a dynamic model may address these concerns3 and offer the conceptual advantage of explicitly testing not just the statistical difference from average but the likelihood of performance falling into a defined poor performance category. Finally, there are systems that use differing criteria for smaller and larger units.4

Ahern and colleagues discuss the potential consequences of poor performance, but omit any reference to exactly who should oversee this process. We assert that the relevant specialty or subspecialty body has a crucial role in overseeing the interpretation of reports. The detection of an outlier is dependent on the nature of the boundaries set for acceptable performance, and the vulnerability of the statistical adjustment model to bias and unmeasured confounders. Such interpretation requires detailed knowledge of the relevant field, an appreciation of the variation between centres, and substantial epidemiological knowledge.