MJA
MJA InSight
MJA Careers
Research

Pneumonia risk stratification in tropical Australia: does the SMART-COP score apply?

Joshua S Davis, Gail B Cross, Patrick G P Charles, Bart J Currie, Nicholas M Anstey and Allen C Cheng
Med J Aust 2010; 192 (3): 133-136.
Abstract

Objective: To examine the performance in tropical northern Australia of SMART-COP, a simple scoring system developed in temperate Australia to predict the need for intensive respiratory or vasopressor support (IRVS) in pneumonia patients.

Design, setting and patients: A prospective observational study of patients admitted to Royal Darwin Hospital in the Northern Territory with sepsis between August 2007 and May 2008. Chest x-rays were reviewed to confirm pneumonia, and each patient’s SMART-COP score was assessed against the need for IRVS.

Results: Of 206 patients presenting with radiologically confirmed pneumonia, 184 were eligible for inclusion. The mean age of patients was 50.1 years, 65% were Indigenous and 56% were men. Overall, 38 patients (21%) required IRVS, and 18 patients (10%) died by Day 30. A SMART-COP score of ≥ 3 had a sensitivity of only 71% for predicting the need for IRVS and 67% for 30-day mortality. As the variables most strongly associated with IRVS were serum albumin level < 35 g/L (odds ratio, 6.8) and Indigenous status (odds ratio, 2.3), we tested a modified scoring system (SMARTACOP) that used a higher weighting for albumin and included Indigenous status. A SMARTACOP score of ≥ 3 had a sensitivity of 97% for IRVS and 100% for 30-day mortality.

Conclusions: The SMART-COP score underestimates the severity of pneumonia in tropical northern Australia, but can be improved by using locally relevant additions.

Risk stratification systems are useful for determining hospital discharge policies and antimicrobial therapy in patients with pneumonia, and identifying patients at risk of adverse outcomes. The best known of these, the Pneumonia Severity Index, was primarily designed to assess the suitability of patients for discharge from hospital.1 This has been shown to perform similarly to or better than other scoring systems, such as the CURB-65.2-4

A recently proposed scoring system, SMART-COP, was developed in Australia and validated in a variety of settings, including patients from North America, temperate Australia and Scotland, but few from tropical areas.5,6 In contrast to previously developed tools, SMART-COP was designed to identify patients requiring intensive supportive care, and its use is being considered for inclusion in version 14 of the Australian Therapeutic guidelines: antibiotic (Jenny Johnstone, Editor, Therapeutic Guidelines Limited, personal communication).

The epidemiology of pneumonia is significantly different in tropical northern Australia than in temperate regions — a high proportion of patients are Indigenous and the average age of patients is lower.7 Furthermore, the pathogens implicated in pneumonia are significantly different to those identified in temperate latitudes.8-10

We examined the performance of the SMART-COP scoring system in patients with community-acquired pneumonia presenting to our referral centre in tropical northern Australia.

Methods

Royal Darwin Hospital (RDH) is the only tertiary referral hospital for the tropical Northern Territory and serves a population of about 150 000 spread across an area of over 500 000 km2. We conducted a prospective observational study of adult patients with sepsis (infection plus at least two criteria for systemic inflammatory response syndrome [SIRS]11) admitted to RDH between August 2007 and May 2008. The study was approved by the Human Research Ethics Committee of the NT Department of Health and Families and Menzies School of Health Research.

Inclusion criteria were at least two symptoms suggestive of pneumonia (new cough, fever, rigors, chest discomfort, new-onset dyspnoea) and a chest radiograph or computed tomography scan taken within 24 hours of admission demonstrating acute pneumonia. All chest radiographs were reviewed by a radiologist and, where the report was inconclusive or ambiguous, radiographs were also viewed by an infectious diseases physician. Exclusion criteria were: immunosuppression,5 active orders limiting life-sustaining treatment, and direct admission to the intensive care unit (ICU).

We defined intensive respiratory support as the need for non-invasive ventilatory support or intubation and mechanical ventilation. Vasopressor support was the use of dopamine, noradrenaline, adrenaline or vasopressin for treatment of hypotension.

We calculated SMART-COP scores as defined in the Australian Community-Acquired Pneumonia Study (ACAPS) (Box 1).5 We used univariate logistic regression analysis to examine the individual components of the SMART-COP score against the need for intensive respiratory or vasopressor support (IRVS). We then assessed the performance of the SMART-COP score and evaluated several variations of it, primarily on the basis of its negative predictive value (NPV), because of the need to identify patients who did not require IRVS and could thus be safely managed in a general ward. Statistical analysis was performed using Intercooled Stata, version 10 (StataCorp, College Station, Tex, USA). A significance level of 0.05 was used.

Results

During the study period, 246 patients were admitted with sepsis and a clinical diagnosis of pneumonia. Of these, 40 did not have radiological evidence of pneumonia and 22 met exclusion criteria (immunosuppression, 10; active orders limiting life-sustaining treatment, 9; direct admission to the ICU, 3), leaving 184 eligible patients (Box 2).

Of the total group, 111 patients (60%) were in the low-risk SMART-COP group (score ≤ 2), and 11 of these (10%) required IRVS (Box 3, Box 4). As a predictor of need for IRVS, a SMART-COP score of ≥ 3 had a significantly lower sensitivity, NPV and area under the receiver operator characteristic curve (AUROC) in our RDH group than in the ACAPS cohort (P ≤ 0.05, Box 5).

Of the SMART-COP score components, new-onset confusion (odds ratio [OR], 22.0; 95% CI, 2.5–194.4) and serum albumin level < 35 g/L (OR, 6.8; 95% CI, 2.9–15.9) were the strongest predictors of the need for IRVS (Box 6). However, confusion was present in only six of the 184 patients (3%), making it less clinically useful in predicting those at risk of needing IRVS. Indigenous status was also associated with the need for IRVS (OR, 2.3; 95% CI, 1.0–5.5). Contrary to expectations, none of the comorbidities that were assessed were significantly associated with the need for IRVS (Box 6).

Given that the SMART-COP score performed poorly at the low-risk end in the RDH cohort, we designed and tested a modified score, SMARTACOP, which increased the weighting for albumin to 2 points and included Indigenous status as a variable (Box 1). These changes improved the performance of the scoring system, largely due to better discrimination of patients in the low-risk group. Of the 111 patients in this group according to the SMART-COP score, 43 were recategorised as moderate risk, of whom 10 (23%) required IRVS. The specificity of both the SMART-COP and SMARTACOP scores was poor (Box 5), and thus they should not be used for their positive predictive value.

The SMART-COP score also had poor sensitivity for predicting 30-day mortality: six patients (5%) with a low-risk SMART-COP score died (sensitivity, 67%), compared with none with a low-risk SMARTACOP score (sensitivity, 100%). The AUROC for 30-day mortality was 0.74 for SMART-COP and 0.77 for SMARTACOP.

Discussion

In this study, we have demonstrated that the SMART-COP scoring system underestimates the need for intensive supportive treatment and the risk of death in a tropical Australian population hospitalised for pneumonia.

Our study population differs substantially from those involved in previous studies of pneumonia risk. Compared with patients in the ACAPS cohort used to derive the SMART-COP score, patients in this study were younger and more likely to be Indigenous, drink hazardous amounts of alcohol, smoke, and have chronic liver disease. In addition, the causative organism in those with an identified pathogen was a gram-negative bacillus in more than half the patients in this study, compared with 18% in the ACAPS study. More than 20% of our patients required IRVS, compared with 11% in the ACAPS study. This may account for a proportion of the decrease in NPV for the SMART-COP score in the RDH cohort compared with the ACAPS cohort, but is unlikely to be solely responsible for the observed difference in NPV.

Low serum albumin level appears to be more strongly associated with the requirement for IRVS in this study than in the original derivation study. Albumin decreases in response to acute inflammation13,14 and thus may be a surrogate marker for late presentation, as well as severity of inflammation. Chronic liver disease and poor nutrition associated with hazardous alcohol intake may also be contributors to low serum albumin levels.

We found Indigenous status to be associated with the requirement for IRVS and with mortality. It is likely that Indigenous status is a surrogate marker for poor health and social disadvantage,15 and these factors are likely to contribute more to poor outcomes than any possible genetic susceptibility. Hazardous levels of alcohol use were present in the majority of Indigenous patients in this study; however, this was not a risk factor for IRVS or mortality. For pneumonia generally, it is likely that an interaction between severe infection and decreased physiological reserve due to multiple underlying comorbidities is what puts an individual patient at risk.

An alternative strategy to improve the performance of the SMART-COP scoring system would be to change the cut-off value used to define low risk. However, changing this cut-off score from 3 to 2 is not as effective as modifying the score; the sensitivity for IRVS changes from 71% to 89% with a cut-off of 2 (compared with 97% at a cut-off of 3 for the modified score), and four of 38 patients who required IRVS would be misclassified using this strategy.

There are several limitations to our study. We only considered patients admitted to hospital; however, it is unlikely that patients not admitted would require IRVS. We also included only patients meeting SIRS criteria, and cannot exclude the possibility that patients without SIRS initially may develop worsening pneumonia later. This study was limited to a single centre, albeit one with the only ICU servicing northern Australia between the Kimberley region in Western Australia and the Queensland border. The number of patients in this study meant that we had limited statistical power to make comparisons or perform multiple logistic regression analysis. The high proportions of Indigenous Australians and of patients with hazardous levels of alcohol use mean that the results of this study may not be generalisable to tropical regions in other countries. We did not collect information on antibiotic use and thus could not control for this in our analysis; however, considering our hospital uses established antibiotic protocols with a high level of staff compliance,7 we would not expect this to vary between groups.

We have demonstrated that the current SMART-COP scoring system does not adequately identify patients requiring IRVS in this tropical setting. We propose some minor modifications that improve its performance, particularly its NPV. We intend to validate this modified scoring system prospectively in patients presenting to the emergency department at RDH.

1 Definition of SMART-COP and SMARTACOP scores*

SMART-COP score

Points


Systolic blood pressure < 90 mmHg

2

Multilobar CXR involvement

1

Albumin level < 35 g/L

1

Respiratory rate raised

1

Tachycardia ≥ 125 beats/min

1

Confusion (new onset)

1

Oxygen low

2

P — arterial pH < 7.35

2

SMARTACOP score

Points


Systolic blood pressure < 90 mmHg

2

Multilobar CXR involvement

1

Albumin level < 35 g/L

2

Respiratory rate raised

1

Tachycardia ≥ 125 beats/min

1

Aboriginal or Torres Strait Islander

1

Confusion (new onset)

1

Oxygen low

2

P — arterial pH < 7.35

2

ACAPS interpretation of SMART-COP score


0–2 points

Low risk of needing IRVS

3–4 points

Moderate risk (1 in 8) of needing IRVS

5–6 points

High risk (1 in 3) of needing IRVS

≥ 7 points

Very high risk (2 in 3) of needing IRVS


CXR = chest x-ray. ACAPS = Australian Community-Acquired Pneumonia Study.5 IRVS = intensive respiratory or vasopressor support. Pao2 = partial pressure of oxygen. Fio2 = fraction of inspired oxygen.
* Calculated using the initial set of observations taken in the emergency department.  Calculated using age-adjusted cut-offs: age ≤ 50 years, ≥ 25 breaths/min; age > 50 years, ≥ 30 breaths/min.  Calculated using age-adjusted cut-offs: age ≤ 50 years, Pao2 < 70 or oxygen saturation ≤ 93 or Pao2/Fio2 ratio < 333; age > 50 years, Pao2 < 60 or oxygen saturation ≤ 90 or Pao2/Fio2 ratio < 250.

2 Characteristics of RDH patients, compared with those from the ACAPS cohort

Characteristic

RDH (n = 184)*

ACAPS (n = 885)*

P


Age in years, mean (SD)

50.1 (16.3)

65.1 (19.9)

< 0.001

Male

103 (56%)

537 (61%)

ns

Indigenous

120 (65%)

10 (1%)

< 0.001

Remote-dwelling

51 (28%)

not reported

Hazardous alcohol use

76/134 (57%)

48 (5.4%)

< 0.001

Chronic renal disease

33 (18%)

169 (19%)

ns

Chronic liver disease

26 (14%)

32 (4%)

< 0.001

Diabetes mellitus

42 (23%)

159 (18%)

ns

Smoker

93/151 (62%)

182 (21%)

< 0.001

COPD

39 (21%)

238 (27%)

ns

Gram-negative pathogen§

31/57 (54%)

72/404 (18%)

< 0.01

Required IRVS

38 (21%)

94 (11%)

< 0.01

Died in the hospital

16 (8.7%)

41 (4.6%)

0.02

Died within 30 days

18 (9.8%)

50 (5.6%)

0.03


RDH = Royal Darwin Hospital. ACAPS = Australian Community-Acquired Pneumonia Study.5 ns = not significant. COPD = chronic obstructive pulmonary disease. IRVS = intensive respiratory or vasopressor support. * All data except age are number (%).  Defined as > 4 standard drinks/day for men and > 2 for women.12  Level of alcohol use not documented for 50 patients and smoking status not documented for 33 patients. § Denominator is those with an identified pathogen from blood cultures or from purulent sputum with predominant growth of an organism seen on Gram stain. Of the 31 gram-negative pathogens in RDH patients, the most common were Haemophilus influenzae (7), Acinetobacter spp. (7), Burkholderia pseudomallei (7), and Escherichia coli (4).

3 Patients in Royal Darwin Hospital cohort requiring IRVS, stratified by SMART-COP and SMARTACOP scores

Number

IRVS

30-day mortality


SMART-COP

Low risk (0–2)

111

11 (10%)

6 (5%)

Moderate risk (3–4)

48

15 (31%)

4 (8%)

High risk (5–6)

22

10 (46%)

6 (27%)

Very high risk (≥ 7)

3

2 (67%)

2 (67%)

SMARTACOP

Low risk (0–2)

68

1 (2%)

0

Moderate risk (3–4)

59

14 (24%)

7 (12%)

High risk (5–6)

39

13 (33%)

4 (10%)

Very high risk (≥ 7)

18

10 (56%)

7 (39%)


IRVS = intensive respiratory or vasopressor support.

4 Risk of needing IRVS according to SMART-COP score in the RDH cohort compared with (A) ACAPS cohort and (B) SMARTACOP score in the RDH cohort


Vertical dashed line indicates cut-off score for low risk (0–2). IRVS = intensive respiratory or vasopressor support. RDH = Royal Darwin Hospital. ACAPS = Australian Community-Acquired Pneumonia Study.5

5 Performance characteristics of the SMART-COP score in the ACAPS and RDH cohorts and the SMARTACOP score in the RDH cohort

SMART-COP score ≥ 3


SMARTACOP score ≥ 3


ACAPS

RDH

RDH


AUROC

0.87 (0.83–0.91)

0.75 (0.66–0.83)*

0.77 (0.70–0.85)

Sensitivity

92% (85%–97%)

71% (54%–85%)*

97% (86%–100%)

Specificity

62% (59%–66%)

69% (60%–76%)

46% (38%–54%)

PPV

22% (18%–27%)

37% (26%–50%)

32% (24%–41%)

NPV

99% (97%–99%)

91% (83%–94%)*

99% (92%–100%)


Data are shown with 95% confidence intervals. ACAPS = Australian Community-Acquired Pneumonia Study.5
RDH = Royal Darwin Hospital. AUROC = area under the receiver operator characteristic curve. PPV = positive predictive value. NPV = negative predictive value. * P ≤ 0.05 compared with the ACAPS.

6 Univariate logistic regression with need for IRVS as dependent variable

Risk factor

No. (%)

OR (95% CI)

P


Systolic BP < 90 mmHg

9 (5%)

2.3 (1.2–4.6)

0.02

> 1 lobe involved on CXR

93 (51%)

2.6 (1.2–5.5)

0.02

Serum albumin level < 35 g/L

82 (45%)

6.8 (2.9–15.9)

< 0.001

Tachypnoea

70 (38%)

1.4 (0.7–2.9)

0.34

Tachycardia

41 (22%)

1.9 (0.8–4.1)

0.13

New-onset confusion

6 (3%)

22.0 (2.5–194.4)

0.005

Hypoxaemia

49 (27%)

1.4 (0.9–2.0)

0.11

Arterial pH < 7.35

8 (4%)

2.0 (1.0–4.2)

0.05

Male

103 (56%)

1.5 (0.8–3.2)

0.23

Indigenous

120 (65%)

2.3 (1.0–5.5)

0.05

Remote-dwelling

51 (28%)

2.0 (0.9–4.2)

0.07

Homeless

17 (9%)

1.1 (0.9–1.1)

0.1

Hazardous alcohol use

76/134 (57%)

1.0 (0.9–1.2)

0.96

Chronic renal disease

33 (18%)

1.6 (0.7–3.8)

0.3

Chronic liver disease

26 (14%)

1.9 (0.8–5.8)

0.17

Diabetes mellitus

42 (23%)

0.9 (0.4–2.1)

0.8

Smoking

93/151 (62%)

0.8 (0.7–1.1)

0.08

Admitted during wet season

126 (68%)

1.0 (0.5–2.1)

0.74

COPD

39 (21%)

1.7 (0.8–3.9)

0.34

Malignancy

7 (4%)

0.6 (0.1–5.4)

0.68


IRVS = intensive respiratory or vasopressor support. OR = odds ratio. BP = blood pressure. CXR = chest x-ray.
COPD = chronic obstructive pulmonary disease.

Received 
29 Apr 2009
accepted 
23 Aug 2009
Joshua S Davis, MB BS, DTM&H, FRACP, NHMRC PhD Scholar,1 and Infectious Diseases Physician2
Gail B Cross, BSc, MB BS, Medical Registrar3
Patrick G P Charles, MB BS, FRACP, PhD, Infectious Diseases Physician,4 and Lecturer5
Bart J Currie, MB BS, FAFPHM, FRACP, Infectious Diseases Physician and Professor in Medicine1,2
Nicholas M Anstey, MB BS, FRACP, PhD, Infectious Diseases Physician and Professor of International Health1,2
Allen C Cheng, MB BS, FRACP, PhD, Senior Research Fellow,1 Infectious Diseases Physician,6 and Associate Professor7
1 Menzies School of Health Research and Charles Darwin University, Darwin, NT.
2 Division of Medicine and NT Clinical School, Royal Darwin Hospital, Darwin, NT.
3 Royal Darwin Hospital, Darwin, NT.
4 Austin Hospital, Melbourne, VIC.
5 Department of Medicine, University of Melbourne, Melbourne, VIC.
6 The Alfred Hospital, Melbourne, VIC.
7 Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, VIC.
Acknowledgements: 

We thank Mark McMillan and Alex Humphrey for data collection and entry; Luke Diolosa and Dianne Stephens for assistance with intensive care databases; and staff in the radiology department for their assistance. This study was funded by the National Health and Medical Research Council (Program Grant 496600, Fellowships to Allen Cheng and Nicholas Anstey, and a scholarship to Joshua Davis).

Competing Interests: 

None identified.

Reference Text: 
Fine MJ, Auble TE, Yealy DM, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med 1997; 336: 243-250.
Reference Order: 
1
PubMed ID: 
8995086
Reference Text: 
Aujesky D, Auble TE, Yealy DM, et al. Prospective comparison of three validated prediction rules for prognosis in community-acquired pneumonia. Am J Med 2005; 118: 384-392.
Reference Order: 
2
PubMed ID: 
15808136
Reference Text: 
Ananda-Rajah MR, Charles PG, Melvani S, et al. Comparing the pneumonia severity index with CURB-65 in patients admitted with community acquired pneumonia. Scand J Infect Dis 2008; 40: 293-300.
Reference Order: 
3
PubMed ID: 
17918017
Reference Text: 
Man SY, Lee N, Ip M, et al. Prospective comparison of three predictive rules for assessing severity of community-acquired pneumonia in Hong Kong. Thorax 2007; 62: 348-353.
Reference Order: 
4
PubMed ID: 
17121867
Reference Text: 
Charles PG, Wolfe R, Whitby M, et al. SMART-COP: a tool for predicting the need for intensive respiratory or vasopressor support in community-acquired pneumonia. Clin Infect Dis 2008; 47: 375-384.
Reference Order: 
5
PubMed ID: 
18558884
Reference Text: 
Chalmers JD, Singanayagam A, Hill AT. Predicting the need for mechanical ventilation and/or inotropic support for young adults admitted to the hospital with community-acquired pneumonia. Clin Infect Dis 2008; 47: 1571-1574.
Reference Order: 
6
PubMed ID: 
18991510
Reference Text: 
Elliott JH, Anstey NM, Jacups SP, et al. Community-acquired pneumonia in northern Australia: low mortality in a tropical region using locally-developed treatment guidelines. Int J Infect Dis 2005; 9: 15-20.
Reference Order: 
7
PubMed ID: 
15603991
Reference Text: 
Douglas MW, Lum G, Roy J, et al. Epidemiology of community-acquired and nosocomial bloodstream infections in tropical Australia: a 12-month prospective study. Trop Med Int Health 2004; 9: 795-804.
Reference Order: 
8
PubMed ID: 
15228489
Reference Text: 
Anstey NM, Currie BJ, Withnall KM. Community-acquired Acinetobacter pneumonia in the Northern Territory of Australia. Clin Infect Dis 1992; 14: 83-91.
Reference Order: 
9
PubMed ID: 
1571467
Reference Text: 
Charles PG, Whitby M, Fuller AJ, et al. The etiology of community-acquired pneumonia in Australia: why penicillin plus doxycycline or a macrolide is the most appropriate therapy. Clin Infect Dis 2008; 46: 1513-1521.
Reference Order: 
10
PubMed ID: 
18419484
Reference Text: 
Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992; 101: 1644-1655.
Reference Order: 
11
PubMed ID: 
1303622
Reference Text: 
National Health and Medical Research Council. Australian alcohol guidelines: health risks and benefits. Canberra: NHMRC, 2001.
Reference Order: 
12
PubMed ID: 
Reference Text: 
Rothschild MA, Oratz M, Schreiber SS. Serum albumin. Hepatology 1988; 8: 385-401.
Reference Order: 
13
PubMed ID: 
3281888
Reference Text: 
Fearon KC, Falconer JS, Slater C, et al. Albumin synthesis rates are not decreased in hypoalbuminemic cachectic cancer patients with an ongoing acute-phase protein response. Ann Surg 1998; 227: 249-254.
Reference Order: 
14
PubMed ID: 
9488524
Reference Text: 
Australian Bureau of Statistics. National Aboriginal and Torres Strait Islander Health Survey, 2004–05. Canberra: ABS, 2006. (ABS Cat. No. 4715.0.)
Reference Order: 
15
PubMed ID: