Influenza vaccine effectiveness in general practice and in hospital patients in Victoria, 2011–2013

Heath A Kelly, Courtney Lane and Allen C Cheng
Med J Aust 2016; 204 (2): 76. || doi: 10.5694/mja15.01017
Published online: 1 February 2016


Objective: To compare influenza vaccine effectiveness in the general practice and hospital settings.

Design: Analysis of annual case test-negative studies.

Setting: Victorian sentinel hospitals and general practices, 2011–2013.

Participants: Patients presenting to general practitioners, or those admitted to hospital with an influenza-like illness who were tested for influenza using a polymerase chain reaction assay. Cases were patients with a positive test result for influenza; non-cases (controls) had a negative test result.

Main outcome measures: Vaccine effectiveness against laboratory-confirmed influenza.

Results: Hospitalised patients were on average older and reported a higher proportion of comorbidities than general practice patients. The pooled estimate of influenza vaccine effectiveness against laboratory-confirmed infection for the 3 years was 50% (95% CI, 26%–66%) for general practice patients and 39% (95% CI, 28%–47%) for patients admitted to hospital.

Conclusions: Influenza vaccines appeared to be similarly modestly effective in the general practice and hospital settings. Influenza vaccination appears to prevent hospital admission by preventing symptomatic infection rather than by attenuating the severity of illness.

The 9th edition of the Immunisation Handbook sponsored by the National Health and Medical Research Council maintained that influenza vaccines were 70%–90% effective in preventing influenza when the match between vaccine strains and circulating strains was good.1 Even when published in 2008, this was probably a generous assessment of the evidence. The 10th edition, published in 2013, maintained that influenza vaccines were 59% effective in preventing influenza in healthy adults and at least as effective in children, although in some years there was no evidence of any benefit.2

Although not explicitly stated in the handbooks, these estimates referred to efficacy in protecting against influenza infections managed in the community, the majority of which are relatively mild. While protection against the mild disease seen in primary care might be modest, it is nevertheless possible that the protection provided against more serious disease, including confirmed influenza infections requiring admission to hospital, might be greater.

In Victoria, two surveillance schemes make it possible to investigate whether there was any major difference in vaccine effectiveness estimates in community and hospital patients. The Victorian Sentinel Practice Influenza Network (VicSPIN) is a group of sentinel general practitioners in Melbourne and regional Victoria, operating since 1997, that has provided estimates of influenza vaccine effectiveness for protection against laboratory-confirmed influenza since 2003.3 The Influenza Complications and Alert Network (FluCAN) is a national hospital-based sentinel surveillance scheme that has provided estimates of influenza vaccine effectiveness since 2010.4 About 40% of patients registered by this scheme were reported by Victorian hospitals.

We compared influenza vaccine effectiveness estimates for 3 years in Victoria, basing our analysis on data from these two sentinel surveillance systems. Each scheme has published separate vaccine effectiveness estimates for the three study years.5-10


We reviewed data for the influenza seasons of 2011–2013 from the general practice and hospital-based schemes. Vaccine effectiveness was estimated by comparing the vaccination status of influenza cases (patients with laboratory-confirmed influenza) with that of non-cases (patients for whom influenza test results were negative). The use of test-negative controls is an established variation of the case–control study design.11

VicSPIN uses a community-based, test-negative design. Sentinel GPs were located in metropolitan Melbourne, Geelong and regional Victoria, and patients were recruited by sentinel GPs when they presented with symptoms consistent with influenza infection. At presentation and before their case status was known, patients were swabbed at the discretion of the GP. Patients with positive influenza test results were defined as cases, and those with negative results as non-cases or controls. Vaccine effectiveness was calculated as 1 − odds ratio (OR), and expressed as a percentage, where the OR compared the odds of vaccination for cases with the odds for controls. Logistic regression was used to adjust estimates for age group (0–17 years, 18–64 years, ≥ 65 years), comorbidity (yes v no), and time within influenza season (number of weeks from peak). Estimates were restricted to patients vaccinated at least 14 days before the onset of symptoms, accepted as the time needed to produce protective antibodies, and to those presenting within 7 days of symptom onset, as data from shedding studies suggest that influenza virus detection declines after 7 days.12 On the assumption that vaccine effectiveness would not be detectable when influenza virus was not circulating, we also restricted our analyses to patients who presented during the influenza season, as defined by positive case ascertainment and sentinel surveillance of influenza-like illness.13 Year was added as a covariate to the regression analysis for the combined 3-year estimate. Although the number of sentinel practitioners participating in the scheme in different years varied slightly, the approach to surveillance remained constant. Vaccination status was determined by patient or GP report, with vaccine date requested as a proxy for a register record. All samples were tested by polymerase chain reaction (PCR) assays at the Victorian Infectious Diseases Reference Laboratory, a designated National Influenza Centre of the World Health Organization. The assay detected influenza A(H3N2), influenza A(H1N1), influenza B and influenza C viruses. Two patients with influenza C were excluded from the analysis.

FluCAN is a sentinel surveillance system that receives data from 17 Australian hospitals. It provides data on the number of cases admitted with severe influenza A or B confirmed by PCR nucleic acid assays in the reporting hospitals’ laboratories. We based our analysis on the four Victorian hospitals reporting to FluCAN (the Alfred Hospital, Monash Medical Centre, University Hospital Geelong, the Royal Melbourne Hospital). Only adults (over 18 years of age) were included in the study. The vaccination status of cases was compared with that of controls (1:1); each selected control was the next patient after each case who presented with an acute respiratory infection and a negative influenza test result. Vaccination was defined as the patient having received the inactivated influenza vaccine at least 14 days before presentation, and the patient’s status was based on patient report and medical record. Vaccine effectiveness was estimated in the same way as for the community patients, but adjusted for potential confounders using conditional logistic regression to account for the matched design. Binary covariates included in the model were: being over 65 years of age, chronic comorbidities, Indigenous Australian status, and pregnancy. Because the control group was frequency-matched by the date of admission using the incidence density control selection strategy, we did not adjust or stratify estimates for time. We conditioned the analysis on the basis of hospital site. For the pooled analysis of all three seasons, the analysis accounted for year by adjusting standard errors with the Huber–White robust sandwich estimator.

During the years included in our study, FluCAN did not collect control data for people aged 0–17 years. We estimated vaccine effectiveness for all ages from the VicSPIN data, but, to improve the comparability of results, we also calculated vaccine effectiveness for the VicSPIN data after excluding the 0–17-year-old age group. All vaccines used in Australia during the study period were trivalent inactivated vaccines. We did not collect information on vaccine manufacturer, and assumed that all vaccines performed equivalently.

Ethics approval for FluCAN data collection and reporting was obtained from the Human Research Ethics Committees of all participating hospitals and the Australian National University. VicSPIN data were collected, analysed and reported under the legislative authorisation of the Victorian Public Health and Wellbeing Act 2008 and the Public Health and Wellbeing Regulations 2009, and therefore did not require formal Human Research Ethics Committee approval.


In the VicSPIN surveillance system, before exclusion of patients vaccinated within 14 days of symptom onset or presenting outside the influenza season, 1680 patients for whom vaccine status by age group was known were available for the three seasons 2011–2013. Their number varied from 354 in 2013 to more than 600 in each of the two earlier years (Box 1). Eighty-five per cent of swabbed patients were from Melbourne or Geelong, the locations of the Victorian sentinel hospitals. Most patients consulting a sentinel GP were aged between 18 and 64 years. Older people were under-represented, but more than 70% were vaccinated each year (Box 1). For the 3 years combined, 5% of patients aged 0–17 years reported a comorbidity, compared with 16% of those aged 18–64 years and 48% of those aged 65 years or more. Comorbidity status was not recorded for 12% of patients.

In the FluCAN surveillance system, 1289 patients were enrolled in Victorian hospitals for whom vaccine effectiveness estimates could be made for the three seasons 2011–2013. The number of participants varied from 271 in 2011 to 737 in 2012. The majority of patients admitted to hospital were at least 65 years old. For the 3 years combined, 76% of adults aged 18–64 years had a comorbidity, compared with 91% of those aged 65 years or more. The estimated vaccine coverage varied during the 3 years between 76% and 82% in those aged at least 65 years, and between 38% and 44% in adults aged 18–64 years.

Information on Indigenous status was not recorded in the VicSPIN data. Fifteen Indigenous patients (nine influenza-positive) were recorded in the FluCAN data. VicSPIN included six pregnant patients, while FluCAN recorded 26 (including 19 who were influenza-positive).

Estimates of protection afforded by influenza vaccines were similar in both schemes. On the basis of the VicSPIN data, vaccine effectiveness against influenza for all age groups managed in general practice varied between 37% in 2011 (when the confidence interval included zero) and 61% in 2013 (Box 2). Vaccine effectiveness estimates changed by no more than four percentage points when the 0–17 years age group was omitted from analysis of the VicSPIN dataset. The pooled estimate for the 3 years was 50% (95% CI, 26%–66%). When the youngest age group was excluded in order to improve comparability with the data for hospitalised patients, the pooled vaccine effectiveness was 51% (95% CI, 27%–67%) (Box 2).

The estimates based on the data from the Victorian sentinel hospitals reporting to FluCAN varied between 35% in 2012 and 52% in 2013, with a pooled estimate of 39% (95% CI, 28%–47%). The crude and adjusted VicSPIN estimates were very similar to those of FluCAN, apart from in 2012. Point estimates were highest in both settings for 2013, and confidence intervals for each estimate included zero in 2011. The point estimates were higher in the general practice than the hospital setting in two of the three years when a significant protection could be demonstrated, but the confidence intervals for the two schemes overlapped in each year. The difference between the pooled vaccine effectiveness in general practice and hospital settings was 12% (P = 0.23).


We found that the estimated protection provided by inactivated influenza virus vaccines, after adjustment for important confounders, was slightly higher in general practice than in hospital-based studies, but with overlapping confidence intervals. All FluCAN and most VicSPIN patients were recruited from metropolitan Melbourne and Geelong; very ill patients from regional Victoria can be transferred to any of the FluCAN hospitals.

Our estimates are similar to those found by other studies of similar design that have used PCR-confirmed influenza as the outcome,14 and suggest that older vaccine effectiveness estimates based on serology data15 or non-specific endpoints16 may have overestimated protection.

By targeting populations at risk of severe outcomes, the national immunisation program has assumed that the vaccine protects against severe outcomes associated with laboratory-confirmed influenza, such as hospitalisation, as well as influenza infections managed in the community. Our data support this view, while the small differences in the point estimates of vaccine effectiveness in community and hospital patients suggest that the influenza vaccine prevents hospital admission by preventing symptomatic infection rather than by attenuating the severity of illness. The difference in vaccine effectiveness may reflect the population at risk of hospitalisation, which includes people more likely to be elderly and to have comorbidities, characteristics that may be associated with impaired vaccine-induced immunity.17

A limitation of this study was the potential for selection bias, given that clinicians (GPs or hospital doctors) had discretion as to which patients were swabbed. However, we have shown there was no association between swabbing and vaccine status in VicSPIN patients during 2011–2014. In an unpublished study of 3649 patients with influenza-like illness who presented to VicSPIN GPs during the influenza seasons of 2011–2014, 2224 samples (64%) were submitted for testing. In the crude analysis, age, sex and year were associated with testing, but vaccination status was not. After adjustment, none of the variables were statistically associated with testing (Lisa McCallum, epidemiologist, Hunter New England Health; personal communication, 20 October 2015). We have not explored this association in the FluCAN dataset.

The selection of patients for inclusion in both VicSPIN and FluCAN distinguish them from surveillance schemes reporting the same outcomes, such as those in the United States18 and New Zealand,19 although the vaccine effectiveness estimates were broadly similar.

Another limitation of our study was that the two surveillance systems collected different covariate data, limiting combined data reporting. Details about covariates have been reported elsewhere.5-10 Influenza subtyping was incomplete in the FluCAN data, but the match between circulating and vaccine strains in the VicSPIN data has been previously explored.20 In all 3 years of our study, circulating influenza A(H1N1) and influenza B strains were matched to the vaccine strains. The influenza A(H3N2) subtype was matched in 2011, but was partially mismatched in the following two years.

A further substantial limitation of this study was that it was underpowered to detect small differences, if they existed, in the two clinical settings.

Vaccine status was incompletely reported in the FluCAN system, but ascertainment has improved in recent seasons. In particular, previous sensitivity analyses using multiple imputation have found similar estimates for vaccine effectiveness when comparing collected and imputed missing data; this suggests that missing data are unlikely to significantly bias estimates of vaccine effectiveness.8 A date of vaccination was provided for at least 85% of VicSPIN patients during this period.

Despite these shortcomings, published results from the VicSPIN studies are consistent with estimates of protection based on meta-analyses of community trial data.21,22 No reviews of efficacy in preventing hospital admission have been published because there have been no trials examining this outcome. However, in schemes that recruit patients from the same defined population in the same year, such as those conducted in Navarra (Spain) and Auckland (New Zealand), vaccine effectiveness estimates have been reported to be similar for hospital and community patients. For instance, vaccine effectiveness in Navarra during 2010–2011 was 75% (95% CI, 61%–84%) in preventing outpatient influenza cases, and 60% (95% CI, 37%–75%) in preventing influenza-associated hospitalisations.23 In Auckland, interim estimates of vaccine effectiveness against laboratory-confirmed influenza for 2014 were 67% (95% CI, 48%–79%) for presentation to a sentinel GP and 54% (95% CI, 19%–74%) for hospitalisation.24 While point estimates of protection in this and in two other studies where patients in community and hospital settings were recruited from the same population were higher for community than for hospital patients, the comparisons are illustrative rather than exhaustive; further, the confidence intervals for vaccine effectiveness estimates overlapped.

A randomised controlled trial of vaccine efficacy in averting hospital admission for laboratory-confirmed influenza might help to resolve the question. To account for annual variation in influenza circulation and vaccine effectiveness, however, this would require a study including tens of thousands patients conducted over more than one influenza season. As well as being extremely expensive, such a trial would not be ethical in view of current recommendations that all people aged 65 years and over, the age group most commonly admitted to hospital for influenza infection, receive the influenza vaccine.4,8-10 This emphasises the importance of observational studies in this context.

The Australian and international surveillance systems provide continuous data that support the effectiveness of the national influenza immunisation program. While the magnitude of benefit may not be as great as earlier studies had suggested, and while variation from year to year is acknowledged, influenza vaccination remains an important intervention for protecting vulnerable patients, as shown by our pooled analyses; this is especially true in the FluCAN setting, where a large majority of patients reported a comorbidity. Further evidence that protection against confirmed influenza infection managed in the community is similar to protection against hospitalisation will require additional studies in which patients in both clinical settings are drawn from exactly the same population.

Box 1 – Vaccination status by age group and case/non-case status for hospitalised and community patients in two sentinel surveillance schemes, Victoria, 2011–2013


Age group

FluCAN*vaccinated patients/all patients (%)

VicSPINvaccinated patients/all patients (%)






0–17 years

2/78 (3%)

4/123 (3%)

18–64 years

19/60 (32%)

51/116 (40%)

9/96 (9%)

58/320 (18%)

≥ 65 years

23/34 (68%)

49/61 (80%)

5/6 (83%)

13/19 (68%)


42/94 (45%)

100/177 (56%)

16/180 (9%)

75/462 (16%)


0–17 years

2/79 (3%)

6/93 (6%)

18–64 years

45/141 (32%)

103/274 (38%)

33/171 (19%)

85/289 (29%)

≥ 65 years

103/153 (67%)

129/169 (76%)

15/18 (83%)

27/34 (79%)


148/294 (50%)

232/443 (52%)

50/268 (19%)

118/416 (28%)


0–17 years

1/17 (6%)

4/50 (8%)

18–64 years

27/106 (25%)

36/84 (43%)

10/59 (17%)

65/199 (33%)

≥ 65 years

26/37 (70%)

44/54 (82%)

0/3 (0%)

21/26 (81%)


53/143 (37%)

80/138 (60%)

11/79 (14%)

90/275 (33%)

FluCAN = Influenza Complications Alert Network; VicSPIN = Victorian Sentinel Practice Influenza Network. * Victorian hospital data only. † Excluded because no controls were available for this age group in these years.

Box 2 – Vaccine effectiveness against influenza, as indicated by hospital admission or presentation to a general practitioner in Victorian sentinel surveillance systems, 2011–2013


Effectiveness against hospital admission with laboratory-confirmed influenza to a Victorian sentinel hospital (FluCAN)

Effectiveness against presentation with laboratory-confirmed influenza to a sentinel general practitioner (VicSPIN)

Crude (95% CI)

Adjusted (95% CI)

Crude (95% CI)

Adjusted (95% CI)

Adjusted (95% CI)*


39% (−1% to 64%)

40% (−6% to 66%)

50% (11%–72%)

37% (−34% to 70%)

35% (−44% to 71%)


18% (−11% to 40%)

35% (8%–54%)

42% (16%–60%)

52% (19%–71%)

53% (19%–72%)


57% (31%–73%)

52% (19%–71%)

67% (34%–83%)

61% (1%–85%)

65% (5%–87%)

Pooled: 2011–2013

34% (9%–52%)

39% (28%–47%)

47% (31%–60%)

50% (26%–66%)

51% (27%–67%)

* Excludes patients aged 0–17 years.

Received 8 September 2015, accepted 13 November 2015

  • Heath A Kelly1
  • Courtney Lane2
  • Allen C Cheng3,4

  • 1 Victorian Infectious Diseases Reference Laboratory, Melbourne, VIC
  • 2 University of Melbourne, Melbourne, VIC
  • 3 Alfred Health, Melbourne, VIC
  • 4 Monash University, Melbourne, VIC



We gratefully acknowledge the contributions to the two surveillance schemes of the following people: Tom Kotsimbos, Paul Kelly, Deborah Friedman, Tony Korman and Louis Irving (FluCAN investigators); Kylie Carville and James Fielding (VicSPIN investigators) and Kristina Grant (VicSPIN data manager). We thank all hospital staff and general practices participating in the two surveillance schemes. FluCAN is funded by the Australian Department of Health. VicSPIN is supported by the Victorian Government Department of Health. Allen Cheng is supported by an NHMRC Career Development Fellowship.

Competing interests:

No relevant disclosures.

  • 1. Australian Government. Department of Health and Ageing; National Health and Medical Research Council. The Australian immunisation handbook, 9th edition. Canberra: Australian Government, 2008. (accessed Nov 2015).
  • 2. Australian Government. Department of Health; National Health and Medical Research Council. The Australian immunisation handbook 10th edition (updated June 2015). (accessed Nov 2015).
  • 3. Kelly H, Carville K, Grant K, et al. Estimating influenza vaccine effectiveness from routine surveillance data. PLoS One 2009; 4: e5079.
  • 4. Cheng A, Tosombis J, Kelly H, et al. Effectiveness of H1N1/09 monovalent and trivalent influenza vaccines against hospitalization with laboratory-confirmed H1N1/09 influenza in Australia: a test-negative case control study. Vaccine 2011; 29: 7320-7325.
  • 5. Fielding J, Grant K, Tran T, Kelly H. Moderate influenza vaccine effectiveness in Victoria, Australia 2011. Euro Surveill 2012; 7: pii=20115.
  • 6. Sullivan S, Komadenis N, Grant K, et al. Influenza vaccine effectiveness in the Victorian influenza season of 2012: influences of waning immunity and vaccine match. J Med Virol 2014; 86: 1017-1025.
  • 7. Carville K, Grant K, Sullivan SG, et al. Understanding protection from influenza vaccine in the community: the Victorian influenza season, 2013. Vaccine 2015; 33: 341-345.
  • 8. Cheng AC, Holmes M, Irving LB, et al. Influenza vaccine effectiveness against hospitalisation with confirmed influenza in the 2010–11 seasons: a test-negative observational study. PLoS One 2013; 8: e68760.
  • 9. Cheng AC, Brown S, Waterer G, et al. Influenza epidemiology, vaccine coverage and vaccine effectiveness in sentinel Australian hospitals in 2012: the Influenza Complications Alert Network (FluCAN). Commun Dis Intell Q Rep 2013; 37: E246-E252.
  • 10. Cheng AC, Dwyer DE, Holmes M, et al. Influenza epidemiology, vaccine coverage and vaccine effectiveness in sentinel Australian hospitals in 2013: the Influenza Complications Alert Network. Commun Dis Intell Q Rep 2014; 38: E143-E149.
  • 11. Foppa IM, Haber M, Ferdinands J, Shay DK. The case test-negative design for studies of the effectiveness of seasonal influenza vaccine. Vaccine 2013; 31: 3104-3109.
  • 12. Suess T, Remschmidt C, Schink S, et al. Comparison of shedding characteristics of seasonal influenza virus (sub)types and influenza A(H1N1)pdm09; Germany, 2007–2011. PLoS One 2012; 7: e51653.
  • 13. Tay EL, Grant K, Kirk M, et al. Exploring a proposed WHO method for defining thresholds for influenza surveillance. PLoS One 2013; 8: e77244.
  • 14. Darvishian M, Bijlsma MJ, Hak E, van den Heuvel ER. Effectiveness of seasonal influenza vaccine in community dwelling elderly people: a meta-analysis of test-negative design case-control studies. Lancet Infect Dis 2014; 14: 1228-1239.
  • 15. Petrie JG, Ohmit SE, Johnson E, et al. Efficacy studies of influenza vaccines: effect of end points used and characteristics of vaccine failures. J infect Dis 2011; 203: 1309-1315.
  • 16. Jackson LA, Nelson JC, Benson P, et al. Functional status is a confounder of the association of influenza vaccine and risk of all-cause mortality in seniors. Int J Epidemiol 2006; 35: 345-352.
  • 17. Goodwin K, Viboud C, Simonsen L. Antibody response to influenza vaccination in the elderly: a quantitative review. Vaccine 2006; 24: 1159-1169.
  • 18. McLean HQ, Thompson MG, Sundaram ME, et al. Influenza vaccine effectiveness in the United States during 2012–13: variable protection by age and virus type. J Infect Dis 2015; 211: 1529-1240.
  • 19. Turner N, Pierse N, Bisselo A, et al. The effectiveness of seasonal trivalent inactivated influenza vaccine in preventing laboratory confirmed influenza hospitalisations in Auckland, New Zealand in 2012. Vaccine 2014; 32: 3687-3693.
  • 20. Kelly H, Sullivan S, Grant K, Fielding J. Moderate influenza vaccine effectiveness with variable effectiveness by match between circulating and vaccine strains in Australian adults aged 20–64 years, 2007–11. Influenza Resp Vir 2013; 7: 729-737.
  • 21. Osterholm MT, Kelley NS, Sommer A, Belongia EA. Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis. Lancet 2012; 12: 36-44.
  • 22. Tricco AC, Chit A, Soobiah C, et al. Comparing influenza vaccine efficacy against mismatched and matched strains: a systematic review and meta-analysis. BMC Medicine 2013, 11: 153.
  • 23. Castilla J, Godoy P, Domínguez A, et al. Influenza vaccine effectiveness in preventing outpatient, inpatient, and severe cases of laboratory-confirmed influenza. Clin Inf Dis 2013; 57: 167-175.
  • 24. Turner N, Pierse N, Huang QS, et al. Interim estimates of the effectiveness of seasonal trivalent inactivated influenza vaccine in preventing influenza hospitalisations and primary care visits in Auckland, New Zealand, in 2014. Euro Surveill 2014; 19: pii=20934.


remove_circle_outline Delete Author
add_circle_outline Add Author

Do you have any competing interests to declare? *

I/we agree to assign copyright to the Medical Journal of Australia and agree to the Conditions of publication *
I/we agree to the Terms of use of the Medical Journal of Australia *
Email me when people comment on this article

access_time 10:41, 10 February 2016
Mark Jones

I have two comments/queries:
1. Was assessment of vaccination status performed blinded to case-control status?

2. Unfortunately this study design carries little clinical relevance, telling us very little about effectiveness in terms of preventing severe outcomes, absolute risk reduction and NNT. It really only tells us the odds of producing a negative PCR test with and without a record of vaccination. The discussion provides a hint of true efficacy where the authors discuss the need to carry out a randomised controlled trial with tens of thousands of participants to enable sufficient statistical power. This suggests the effect of vaccine is very small.

Competing Interests: No relevant disclosures

Dr Mark Jones
University of Queensland

Online responses are no longer available. Please refer to our instructions for authors page for more information.