Propensity score weighting for addressing under-reporting in mortality surveillance: a proof-of-concept study using the nationally representative mortality data in China

The China Disease Surveillance Points System

The DSP was initiated in 1978 and adjusted three times in 1990, 2005 and 2010 on the
basis of economic development, geographic location, Gross Domestic Product (GDP),
proportion of non-agricultural population and the total population of the country
to ensure representativeness. After adjustment in 2010, the DSP system included 64
urban and 97 rural surveillance sites in all 31 provinces (autonomous regions and
municipalities) covering seven percent of the total population in China. The information
provided by the system can be used to estimate causes of death among the national
population and the detailed description of DSPs has been published elsewhere 2], 6]. In brief, all deaths were reported in the monitoring stations in the hospitals,
community health centers and village clinics in each DSP based on death certificates.
Data on demographics, date of death, place of death, cause of death, and main symptoms
and signs (for verbal autopsy), etc., were collected. The 161 DSP-level and 31 provincial-level
Centers for Disease Control and Prevention (CDC) were responsible for data quality
through regular checking, supervision, feedback and verification. Starting in 2008,
all the deaths in DSPs were reported through an online death causes monitoring system.

Survey of the under-reporting death cases in China

To address the under-reporting, periodic evaluations for completeness of registration
were conducted once every three years in DSPs. Two under-reporting field surveys have
been carried out during the period 2006–2008 and 2009–2011 respectively. The survey
in 2006–2008 showed that the national total crude rate of under-reporting was 16.7 %
and the weighted rate was 17.4 %; the under-reporting rate for children aged 5 years
and below (35.0 %) was much higher than that for people above age 5 (16.9 %) 7].

Field survey design

An under-reporting survey was conducted in all 161 DSPs from July to October in 2012.
Within each DSP, three townships (in rural areas) or streets (in urban areas) whose
crude death rate (CDR) was close to that DSP’s average CDR were first selected as
candidate fields for the under-reporting survey. One township/street was finally chosen
as the field site if its economic level was similar to the DSP’s average and the population
size was in the middle level among all the townships/streets in the DSP. All the residents
in the selected township/street were included as the survey population. Deaths occurring
from January 1
^st
, 2009 to December 31
^st
, 2011 in the families were investigated using interviews with the surviving household
residents. The information of death population collected in the field survey included
demographics, death-related information such as causes of death, highest level of
hospital where illness was diagnosed, and diagnostic basis.

Data collection

A list of decedents from the focal time period was created for each resident group
(the smallest administrative unit) within all villages and communities in the selected
townships or streets by recall of the resident group leaders. The initial list was
checked and complemented by data from public security departments, civil affairs departments,
family planning departments, and maternal and child health departments. Using the
final list of deaths, the interviewers in each village or community surveyed each
family which experienced a death to verify and revise relevant information on the
death records.

Identification of missed deaths

Death records between the field survey system and the routine online death cause surveillance
system in each DSP were first matched by an automatic computer checking algorithm.
Persons included in both systems were identified as a match when national ID matched.
If the national ID was missing, persons with the same name, gender and age (within
three years) were used to identify a match. After an initial computer matching process,
all mismatched cases were checked and verified by a further manual checking in the
DSP level. The local staff checked each mismatched case with the records from the
surveillance system. Missed death cases were identified after this thorough manual
verification.

Statistical methods

To test the conformity between under-reporting field survey data and the dataset of
DSP system, we used a test of goodness of fit to calculate and compare the frequency
distribution of main variables (age, cause of death, highest level of hospital where
illness was diagnosed, and diagnostic basis) of the two datasets. The highest level
of hospital where disease was diagnosed and the diagnostic basis were important indicators
for accuracy of the underlying cause of death. Hospitals at the township-level and
above were generally regarded as qualified to make correct diagnosis and the diagnoses
made at village hospitals were checked and verified by senior DSP staffs. The diagnosis
was considered reliable if it was made based on symptoms/signs, physio-biochemistry,
pathology, autopsy or surgery. Inference-based diagnosis were verified with the original
investigation documents.

We described the detailed steps of PS and CMR method as follows:

Propensity score weighting method

We used a propensity score weighting method based on a logistic regression of under-report,
where the variables were selected stepwise. The inclusion criteria and the exclusion
criteria were 0.1 and 0.12 respectively. The variables used for analysis included
age, gender, rural/urban residency, geographic locations, educational attainment,
occupation, marital status, cause of death, place of death and diagnostic unit. Geographic
locations were classified as east, central and west according to criteria of National
Bureau of Statistics. The cause of death was identified according to the International
Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10).

We used two groups (those aged 5 years and below and those above 5 years) to set up
two separate models. The model included age, geographic location, urban and rural
for children aged 5 years and below. Whereas for those over 5 years old, the model
included age, gender, geographic location, occupation, rural/urban residency, marital
status, place of death, diagnostic unit, cause of death and year of death. Propensity
score weighting integrated the information of several major covariates into one propensity
score variable. The estimated propensity score weighting may lead to a substantial
reduction in bias, especially for small groups. The analytical procedure is as follows:

Step 1: Model estimation

The sampled under-reporting survey may not be perfectly representative of the whole
DSP in terms of socioeconomic variables that are related to the probability a death
is included in DSP. We applied logistic regression to the sociodemographic variables
to predict the probability a respondent was included in the routine surveillance in
the sampled under-reporting survey site, using all individual records in the under-reporting
field survey of 2009–2011 as the gold standard. We used age, sex, place of death and
other predictor variables in the model. The coefficient and standard error for each
variable of the models are shown in Table 1 (for under 5 years) and Table 2 (for above 5 years). The goodness of fit reached 0.208 and 0.214 respectively. The
regression equations for the two models were:

Table 1. Coefficient and standard error of variables in model 1 (for under 5 years)

Table 2. Coefficient and standard error of variables in model 2 (for above 5 years)

Equation for model 1 (under 5 years):

where x1 refers to urbanity, x2 refers to age group, x3 refers to year and x4 refers to the highest level of hospital
where disease was diagnosed listed in Table 1.

Equation for model 2 (above 5 years):

where x1 refers to region, x2 refers to age group, x3 refers to year, x4 refers to the highest level of hospital
where disease was diagnosed, x5 refers to marital status, x6 refers to education,
x7 refers to occupation, x8 refers to place of death and x9 refers to cause of death
listed in Table 2.

Step 2: Weighted estimates for death cases

The probability of being reported for each observation (p
_i
) was based on the logistic regression model of the field survey data. Weights for
each case were calculated as w
_i
?=?1/p
_i
. The weighted number of deaths from 2009 to 2011 (Ts) was:

Where N
_s
is the total number of death cases from the DSP 2009–2011 surveillance.

Theoretically, the sum of w
_i
of the cases represented the actual number of deaths, which was the total number of
deaths that occurred during 2009–2011.

Step 3: The under-reporting rate of DSP from 2009–2011 (P) based on propensity score
weighting was:

CMR method

To compare the results calculated from propensity score weighting method, we also
used the CMR method to calculate the under-reporting rate. CMR has been widely used
in wildlife science to estimate the size of free-living animal population and it has
been advocated for use in estimating completeness of a registration 8]. In the two-sample capture-mark-recapture approach, an estimate of the true population
size is derived assuming independence of ascertainment by evaluating the degree of
overlap from existing data sources.

To perform CMR analysis, the estimated overall death toll (N) was

where M is defined as the total number of cases in the routine DSP surveillance, n
is defined as the total number of cases in under-reporting field survey, and m is
defined as the number of cases reported in both systems.

The under-reporting rate of DSP from 2009–2011 (p) based on CMR was: