CareTrack: assessing the appropriateness of health care delivery in Australia

In reply: The CareTrack Australia study1 has received support for its general recommendations, but also some criticism. Here, we address the issues raised by commentators.

Levels of evidence: Phan and Srikanth, and Mackay join previous commentators2,3 in calling into question the key CareTrack finding — that adult Australians received appropriate care at 57% of eligible health care encounters in 2009 and 20101 — because 71% of indicators were supported by consensus-based recommendations,2 and only 15% were backed by National Health and Medical Research Council Grade A or B recommendations.3 In fact, compliance with indicators supported by high grades of recommendations or levels of evidence was no different to overall compliance. It was 54% (95% CI, 49%–60%) for 6431 eligible encounters backed by Grade A or B recommendations, 56% (95% CI, 43%–70%) for 4551 eligible encounters backed by Level I or II evidence, and 62% (95% CI, 58%–66%) for 20 875 eligible encounters backed by consensus-based recommendations.

Choice of indicators: In an earlier letter to the Journal in support of using primary care encounters for smoking, nutrition, alcohol and physical activity interventions (with which we strongly agree), Wright and Bolam state that CareTrack failed to consider these “core aspects of preventive care”.4 However, this is not the case. To be enrolled, participants had to have one or more of the 22 study conditions. Indicators relating to smoking, nutrition, alcohol intake and physical activity were included, where relevant, within specific conditions (see Appendix 1 in our article; online at mja.com.au1). For example, 10 indicators across eight conditions addressed the assessment of smoking status or advice to stop smoking (eg, “Patients with a new diagnosis of a stroke/[transient ischaemic attack] that smoke were counselled to stop smoking”). Average compliance with the 10 smoking-related indicators was 79% (95% CI, 65%–93%). Compliance with other health promotion indicators will be outlined in future publications. Furthermore, questions in these areas have been included in the next stage of our research, which is currently underway and involves direct surveys of CareTrack participants.

Phan and Srikanth point out that CareTrack did not use outcome indicators and object to the use of the outdated ABCD2 tool for assessing stroke risk. In support of this, they cite their own work published within the past 2 years. First, CareTrack is not an outcome study. We focused on process so that the results can inform improvement initiatives.1 Second, as CareTrack indicators for stroke were ratified by clinician experts for care expected in 2009–2010, the 2007 guidelines were the most appropriate source.5 These state unequivocally that the ABCD2 tool should be used at presentation in both primary and secondary care settings (a statement backed by a Grade B recommendation and Level II evidence). Phan and Srikanth point out that the use of out-of-date guidelines to develop indicators is problematic. We agree, and this is one of the important recommendations arising from our study — that a process be put in place for ensuring guidelines are up to date.6

Bias and attrition: The issues of bias arising from our sampling methods and the high attrition of potential participants were identified.2 We have already discussed these.1 When results were adjusted to account for demographic differences and non-response bias, the findings were virtually identical, which should allay these concerns (see Appendix 2; online at mja.com.au1).

The small numbers of participants for some conditions: This is an inevitable consequence of the population-based sampling we used. We agree that results from small sample sizes with wide confidence intervals should be viewed with great caution.2 However, CareTrack findings for these conditions are consistent with other studies, as noted by commentators elsewhere.7

Effects of comorbidities: Commentators raise the issue of the potential effect of comorbidities on compliance.2,3 Comorbidities are not a consideration for most indicators; where they may be, the indicators take account of virtually all contingencies (eg, see indicators 129, 428–4301), rendering insignificant any effect on overall compliance.

Dawda suggests that we should learn from the CareTrack high performers. We agree. We are currently examining what drives health care decision making from the perspectives of both health care providers and recipients, and are taking particular note of what makes the high performers “get it right”.

Scott and Del Mar also commented on the issues of care received but not documented, the paucity of overuse indicators, and inter-rater reliability.2 All these have already been addressed.1,6

Finally, Ackermann3 and Mackay point out that CareTrack did not assess the quality of primary care practices. We agree. This was not the aim of the study. Our study simply aimed to determine the proportion of health care encounters at which adult Australians received appropriate care, regardless of the context in which it was delivered.

A new finding that seems to have emerged from our interactions with the expert commentators is that some commentators attach a weight to evidence-based recommendations that our findings demonstrate is not reflected in everyday practice. This reinforces our recommendation that much more work is needed on translating evidence into practice.6