To our knowledge, this is the first formal evaluation of data sources for estimating
antenatal influenza vaccinations using self-report as the “gold standard.†We found
that systems which rely on provider-reported vaccination events or electronic medical
records poorly capture influenza vaccinations administered to pregnant women, and
the validity of these systems varies widely. A state-wide antenatal influenza vaccination
database which relies on passive reporting from immunisation providers accurately
recorded 46 % of influenza vaccinations administered to pregnant women. Electronic
medical records within a public maternity hospital recorded 67 % of antenatal vaccinations,
and public health service records recorded 29 %. These results indicate there is significant
under-reporting of vaccinations administered to pregnant women in all systems.
The low sensitivity and negative predictive values identified in our study are perhaps
not surprising, considering these systems rely on passive reporting from either immunisation
providers or hospital staff. Although surveillance systems which rely on passive reporting
have methodological advantages, such as low cost and relatively simple implementation,
they often have low sensitivity and may not be representative. In the case of maternal
vaccinations, it is apparent that immunisations are not comprehensively reported and
entered, as demonstrated by our study. Previous research has demonstrated that incentives
can be used to improve the timeliness and accuracy of recording vaccination events,
and this may be one method for improving the validity of these systems 13], 14]. However, this introduces additional resource requirements for sustaining this surveillance
activity. Education and targeted intervention may also improve recording; for example,
in our setting, only one-third of vaccinations administered by a general practitioner
were recorded in the state vaccination database. Targeted education could help improve
provider reporting from these sites.
Vaccinations administered outside of traditional providers, such as places of employment,
were also infrequently recorded in these systems, likely because these providers are
unlikely to report vaccination events to the state government and are less often included
in hospital records, as observed by our study. Previous investigations have shown
that approximately 30 % of working-age adults receive influenza vaccines in non-traditional
healthcare settings 15], and nine percent of vaccinated women in our study reported having been vaccinated
at their workplace. Considering 13 % of immunisations provided outside of general
practice or public hospital clinics were recorded by the WAAIVD and 29 % were recorded
in the maternity hospital database, this is likely a large factor in the under-reporting
of vaccinations to these systems and an area for improvement.
In the absence of a comprehensive system for monitoring adult vaccinations, such as
an adult vaccination register, influenza vaccinations are difficult to routinely monitor
in all adults. However, unique to antenatal vaccinations, there are two logical time
points available for recording influenza vaccinations during pregnancy: once at vaccine
administration, as the WAAIVD is structured, and again at delivery, as recorded by
hospital databases. Data collection at the time of delivery can be logistically convenient
as there are statutory requirements for the collection of information related to the
pregnancy and birth 16]. Under the Health Act 1911, midwives are required to notify the Department of Health
of the outcomes of all birth events within 28Â h of birth by completing a Notification
of Case Attended form 16]. Data collected in these forms are used to establish the Midwives Notification System,
a state-based perinatal data collection 17]. The inclusion of influenza vaccination status in state or national perinatal data
collections would establish an annual, electronic source of estimating influenza vaccine
uptake in pregnant women. Other countries such as Canada choose to monitor influenza
vaccination in this way. For example, the Nova Scotia Atlee Perinatal Database, a
population perinatal database collects data on maternal health, details on the delivery,
and information related to influenza vaccination 18]. Data are collected after hospital discharge and are based on standardised clinical
forms and hospital records. These databases have proven useful for evaluating uptake
and the effectiveness of maternal immunisation programs and could be used for similar
purposes in Australia.
Globally, there are three types of systems which have been used to estimate influenza
vaccine uptake during pregnancy: 1) population surveys; 2) healthcare utilisation,
insurance claims, and pharmaceutical dispensary data; and 3) vaccination registries.
Population surveys are a valid and reliable method for estimating uptake 5]–7] and have been previously used for surveillance of maternal influenza immunisations
in Australia 4]. However, they are time-consuming, resource intensive, and can be difficult to implement
annually. Health service utilisation and health insurance claims databases have also
been used for surveillance of maternal influenza immunisation. The UK General Practice
Research Database, a primary care database containing de-identified health records
from 8.4 % of the UK population, has been used to determine the proportion of women
immunised against influenza 19], 20]. In the United States, patient-specific insurance claims data, such as Kaiser Permanente
health plan membership data 21], 22] and LifeLink
TM
Health Plan Claims Database 23] have been used to evaluate uptake of influenza vaccine during pregnancy. In France,
a database of prescription use in the general population has been used to evaluate
risks associated with drugs administered during pregnancy 24], 25]. While these databases tend to draw from a large population of unique members and
have produced estimates similar to those of population surveys 18], 20], such databases are not necessarily designed to provide accurate vaccination uptake
estimates. National databases or registries, such as the national database in Denmark
which was established during the H1N1 pandemic to monitor H1N1 vaccination 26], are an ideal source of vaccination uptake data. However, the Denmark registry was
restricted to pandemic vaccinations, and would need to be maintained annually for
it to be useful in monitoring seasonal influenza vaccine uptake in pregnant women.
Electronic vaccination registries of the whole population, such as those of Denmark
and a number of other countries in Europe 27], 28], could be used to collect seasonal influenza vaccination information needed to evaluate
vaccination programs, in pregnant women and other target groups. Australia’s electronic
immunisation registry is currently restricted to children under the age of seven years,
and the case for a whole-of-life immunisation registry in Australia has been argued
in the past 29], 30]. Expansion of this registry to adults would allow for monitoring of influenza vaccination
during pregnancy as well as other target groups. Previous research has shown that
electronic vaccination registries estimate vaccination status similar to self-reported
vaccination status for women and high risk groups 31], 32], and electronic information is likely the most efficient source of routine vaccination
information. Recent establishment of antenatal pertussis vaccination programs in the
United Kingdom and the United States of America, and their imminent introduction in
Australian jurisdictions underscore the importance of national surveillance systems
for adult, including antenatal, vaccination uptake.
There are several limitations to consider when interpreting our data. First, we used
self-reported vaccination status as the “gold standard†in this evaluation. While
self-report has historically been proven to be a good measure of vaccination status,
it could be argued that this is an imperfect measure. In our study, the majority of
self-reported vaccinations could be verified by the immunisation provider (80.6 %);
however, influenza vaccines administered in the workplace or by non-traditional providers
could not be verified. Published literature support the validity of self-reported
vaccination status in adults and indicate that false negative self-reports are extremely
rare. The sensitivity of self-report has previously been shown to range from 90–100
% 9]–11], and self-reported vaccination status is commonly used to estimate influenza vaccine
coverage 33], 34]. However, given 20 % of self-reported records could not be verified, it is possible
that some self-reported vaccinations were inaccurately classified as ‘true.’ Second,
hospital A routinely offers influenza vaccination to its patients and reports these
to WA Health. Hospital A is responsible for the majority of deliveries in public hospitals
in Western Australia. It is likely that a large proportion of women who reported a
public hospital clinic as their antenatal care provider were immunised at hospital
A and were more likely to be recorded. Third, Hospital A and health service B provide
maternity services within the metropolitan area of the state, and so the sample of
women in this evaluation may not be representative of pregnant women across Western
Australia. Finally, it is possible that a name and date of birth search in both databases
was insufficient to identify women, which may lead to some discrepancies in the data.