healthmedicinet

By systematic literature search we identified 3385 publications, of which 23 were
finally included in our analysis (Fig. 1) 3], 5], 7], 15]–33]. Details on the excluded studies are reported in Additional file 3. Baseline characteristics
of the 23 included studies are shown in Table 2. Of these, 20 were cohort studies, while the remaining three had a case–control design.
The studies were conducted in North America (?=?14), Europe (?=?6), Taiwan (?=?2) or in multiple continents (?=?1) and mainly used disease classification codes (e.g., ICD-9) or civil register
data for the identification of outcomes. In three studies interviews were conducted
or self-administered questionnaires were applied to collect primary data on relevant
outcomes or vaccination status 24], 25], 27]. Except of four studies, which were either performed in students (?=?1), in adults aged 40+ years (?=?1), or in women who recently experienced live birth (?=?2), all studies were conducted in elderly persons. Seven studies covered populations
with underlying comorbidities, namely with (chronic) heart disease (CHD), 21], 22] end-stage renal disease (ESRD), 17], 23] chronic obstructive pulmonary disease (COPD), 28], 29] or patients with diabetes or vascular disease 33].

Fig. 1. Flow chart for the systematic review

Table 2. Baseline characteristics of included studies

Reported outcomes

The included studies reported VE estimates (crude and adjusted in-season plus adjusted
off-season estimates) related to 11 different clinical outcomes: all-cause mortality
(?=?12 studies), death due to respiratory event (?=?2), major adverse vascular event (?=?1), hospitalization due to influenza and/or pneumonia (?=?7), hospitalization for acute coronary syndrome (?=?1), influenza-like illness (?=?3), cardiac death (?=?1), hospitalization due to cardiovascular disease (?=?1), prematurity (?=?1), small for gestational age (?=?1), and medically attended respiratory infections in infants (?=?1). None of the clinical outcomes was required to be confirmed by laboratory testing
for influenza viruses.

Risk of healthy vaccinee bias and confounding by indication

Of the included 23 studies, 19 (83 %) showed a high risk of bias (either healthy vaccine
bias, confounding by indication, or both). Two studies we judged to be at high risk
of healthy vaccinee bias but not confounding by indication (Table 3). One of these studies was performed in patients with end-stage renal disease, with
vaccinated participants having more favorable prognostic markers than unvaccinated
participants; 23] the other study covered patients suffering from COPD and indicated that vaccinated
patients had less (severe) comorbidities as indicated e.g., by the Charlson comorbidity
index, when compared to unvaccinated patients.

Table 3. Risk of healthy vaccinee bias and confounding by indication in the included studies,
as judged from the baseline characteristics of vaccinated and unvaccinated participants

Fourteen studies showed a high risk of confounding by indication, but not of healthy
vaccinee bias. In 13 of them, 3], 15], 16], 18], 19], 21], 22], 24], 28], 30]–32], 34] this was indicated by a significantly higher proportion of vaccinated patients with
comorbidities (compared to unvaccinated participants), whereas in one study 20] medical visits served as indicator. In three studies, we found indication for both
types of bias/confounding occurring simultaneously 7], 17], 33]. In these studies, the group of vaccinated participants had a higher proportion of
comorbidities, while at the same time unvaccinated participants showed a higher proportion
of functional impairments or other relevant comorbidities. In a further three studies,
5], 25], 27] no major differences in baseline characteristics between vaccinated and unvaccinated
study participants were found. In the remaining one study, risk of bias was unclear
due to unclear data and reporting (Table 3) 26].

Adjustment for confounders and impact on point estimates

In ten of 12 studies reporting on all-cause mortality, adjustment for confounders
increased the estimate of VE. The same effect of adjustment was observed in all studies
reporting on hospitalization, major adverse vascular events, influenza-like illness
and cardiac death. For the remaining outcomes, the effect was either very small or
adjustment decreased the VE estimate. All studies adjusted at least for age and comorbidities,
although definitions of the latter differed between individual studies (Table 4).

Table 4. Crude and confounder-adjusted estimates of vaccine effectiveness during the influenza
season in the included studies

We pooled the data for the outcomes all-cause mortality, hospitalization due to influenza
or pneumonia, and ILI since more than one study reported on these outcomes. For all-cause
mortality, this ratio of odds-ratio analysis indicated that adjustment for confounders
increased the effect of vaccination by 12 % (95 % CI: 7–17 %) (Fig. 2a). For hospitalization due to influenza or pneumonia, effect size increased by 9 %
(95 % CI: 4–14 %) after adjustment for confounders (Fig. 2b). For the outcome ILI, adjustment for confounders increased VE estimate by 7 % (95
% CI: 4–10 %).

Fig 2. Impact of adjustment for confounders, expressed as ratio of odds ratios (crude/adjusted):
(a) All-cause mortality, (b) Hospitalization due to influenza or pneumonia

Off-season estimates

The included 23 studies reported a total 31 off-season estimates. Three of the studies
reported pre-season as well as post-season estimates 7], 20], 35]. Two studies reported only pre-season estimates, 5], 23] while five studies provided data on post-seasons “effectiveness” only 15], 16], 18], 19], 32]. The remaining studies reported off-season estimates either for the whole period
outside the influenza season or for single months before and after the seasons. Most
studies defined beginning and end of influenza periods according to national influenza
surveillance data. If more than one off-season estimate was provided, we decided to
use the post-influenza season estimate for analysis (for a detailed description of
the definition of “off-season” in the studies, see Additional file 4).

Analyzing the 31 adjusted off-season estimates that were reported by the 23 included
studies, we found statistically significant effects of influenza vaccination outside
the influenza season in 13 studies (Figs. 3 and 4). Nine (39 %) of the 23 included studies reported at least one statistically significant
VE estimate outside the influenza season (Figs. 3 and 4). These off-season effects were not restricted to the outcome all-cause mortality,
but were also reported for four other outcomes (major adverse vascular events, hospitalization
due to influenza/pneumonia, acute coronary syndrome, ILI). However, significant off-season
estimates were more likely to occur when all-cause mortality was used as an outcome
(8/13; 67 %) compared to other outcomes (5/19; 26 %; p?=?0.03 by chi
²
test). We then evaluated whether the occurrence of significant off-season estimates
was related to the risk of healthy vaccinee bias, as judged from the baseline data
of the respective study populations. We found that 46 % (6/13) of the significant
off-season estimates were associated with high risk of healthy vaccinee bias at baseline.
In contrast, only 6 % (1/18) of non-significant off-season estimates were associated
with high risk of healthy vaccinee bias (p?=?0.01 by chi
²
test). Studies covering non-elderly populations did not report statistically significant
off-season estimates for neither outcome.

Fig. 3. Odds ratios (95 % CIs) of influenza vaccine effectiveness during influenza seasons
(black square), during pre-influenza seasons (striped circle) and post-influenza seasons
(white circle) against all-cause mortality (a), death due to respiratory event (b), death due to cardiac event (c), and major adverse vascular event (d)

Fig. 4. Odds ratios (95 % CIs) of influenza vaccine effectiveness during influenza seasons
(black square), during pre-influenza seasons (striped circle) and post-influenza seasons
(white circle) against hospitalization due to influenza or pneumonia (a), hospitalization for acute coronary syndrome (b), hospitalization due to cardiovascular diseases (c), influenza-like illness (d), prematurity (e), small for gestational age (f), and medically attended respiratory illness in infants (g)