Using pay for performance incentives (P4P) to improve management of suspected malaria fevers in rural Kenya: a cluster randomized controlled trial

Study facilities

All study activities took place between June 2012 and December 2013. Baseline data
was collected in September and October 2012. The intervention was launched in October
2012 and continued until October 2013.

In total, 18 facilities were enrolled and 17 facilities (nine intervention and eight
comparison) completed the 1-year intervention phase of the study. One facility was
excluded from the analysis due to extended absence of a laboratory technologist and
inability to provide laboratory diagnosis of malaria. The remaining facilities completed
the study and were included in the analysis (Fig. 1). Overall, 14,721 patient encounters (including 5,584 slide-negative children aged
1–5 years) were included in the patient-level analysis.

There were no significant differences in demographics of the patients attending facilities
or the slide positivity rate in each arm (Table 2). Comparison facilities had higher baseline patient volume in the laboratory than
intervention facilities, but the difference was not statistically significant (P?=?0.24).

Table 2. Facility and patient characteristics in each arm

Impact of interventions on facility performance

Raw proportions, unadjusted odds ratios (ORs) and adjusted ORs from the mixed-effects
logistic regression model for the prescription of AL to malaria-negative and malaria-positive
patients in quarter 4 compared to baseline are shown in Table 3. Full model results including coefficients of all covariates can be found in Additional
file 3.

Table 3. Mixed-effects logistic regression of AL use by malaria status in quarter 4 compared
to baseline. Comparison of AL treatment in intervention and control arm for all patients
and by sub-group (transmission zone, age group)

By the final quarter of the intervention period, prescription of AL to malaria-negative
patients in the intervention arm dropped from 22.4 % at baseline to 7.3 % (112/1,526)
in quarter 4 compared to a decline of only 5.4 percentage points in the intervention
arm (16.3 % to 10.9 %). After correcting for individual and facility covariates, the
odds of receiving AL following a negative test in the intervention arm was 0.15 in
quarter 4 compared to baseline and 0.42 in the comparison arm relative to baseline,
indicating a larger improvement in the intervention arm over the study period. This
translates to a 2.75-fold greater reduction in odds of inappropriate prescription
of AL in the intervention arm relative to the comparison arm from baseline to the
end of the study.

Facilities in high transmission areas had ten times higher odds of AL prescription
to negative patients than those in low transmission areas (adjusted OR?=?10.36, 95 %
CI: 4.96–21.64; Additional file 3), therefore we further stratified our analysis on transmission zone (Table 3). Intervention and comparison facilities in low malaria transmission areas had equally
low rates of AL prescription to malaria-negative patients by the end of the study
period, but intervention facilities improved more relative to baseline than comparison
facilities. In high transmission areas, facilities in both arms dispensed AL to 35 %
of malaria-negative patients at baseline. By the end of the study, intervention facilities
again showed more improvement than comparison facilities (67 % reduction compared
to 50 % reduction in raw proportion, ratio of adjusted OR?=?0.45, 95 % CI: 0.28–0.74).

Malaria-negative children under 5 years of age were more likely to get AL than malaria-negative
adults in both arms and the odds of a malaria-negative child receiving AL was even
higher when diagnosed by microscopy compared to RDT (adjusted OR?=?0.77, 95 % CI:
0.60–0.99; Additional file 3).

At baseline and endline, there was no significant difference in prescription practices
to malaria-confirmed cases between the intervention and comparison groups. AL dispensed
to laboratory-confirmed malaria cases improved slightly, but not significantly, in
both arms by the fourth quarter of the study.

The rate of prescription to patients without a test was evaluated at the cluster level.
The percent of AL given to patients without a test was 25.7 % in the intervention
arm at the end of the study compared to 40.3 % in the control arm. After adjusting
for transmission zone and volume of patients seen at a facility, the rate of AL prescriptions
to patients without a test was 5-fold lower in the intervention arm (Incidence rate
ratio (IRR)?=?0.18, 95 % CI: 0.07–0.48; Table 4).

Table 4. Effect of intervention on diagnostic testing prior to treatment with AL

Intervention outcome

The maximum amount that could be earned by an intervention facility over the 1-year
study period was 400,000 KSh (approximately $5,000 USD). The mean quarterly amount
earned was $668 USD, approximately half of the total possible amount. There was considerable
variation between facilities in the amount earned, indicating that individual facility
characteristics or context were important (by ANOVA, 78 % of variance is explained
by between-facility variation and 22 % within-facility, across quarters).

The indicator for which intervention facilities showed the most improvement was the
percent of malaria-negative patients who received AL (22.4 % at baseline down to 7.3 %
in Q4; Table 5) and percent of patients given AL without a malaria test (41 % down to 26 %). However,
the latter still remained high at approximately 26 %. Laboratory performance indicators
steadily improved each quarter. Quarterly performance is also shown for the comparison
arm.

Table 5. Quarterly performance of facilities for incentivized indicators

Twenty-seven and 39 % of incentive funds were spent on laboratory equipment and patient
equipment, respectively, including microscopes, delivery beds, centrifuges, steam
sterilizers, biosafety cabinet, blood pressure cuffs and minor surgical tools. Seven
percent was used for facility repairs such as plumbing, electrical, fencing, security
grills and doors. Eleven percent was used for laboratory reagents for urinalysis,
blood grouping, blood sugar testing, hemoglobin measurement and other medical consumables.

Pre-study to post-study differences in clinical diagnosis and AL prescription

Individual patient observations describing malaria testing and treatment were not
available prior to the study. Therefore, to observe change in behavior before and
after study training (microscopy training and 1-day workshop to review the malaria
treatment guidelines with clinicians) compared to the intervention period, we extracted
monthly facility-level data from the Kenya Health Information System. The ratio of
total monthly cases of malaria diagnosed clinically (without parasitological confirmation)
to total outpatients, and the ratio of AL courses dispensed to total outpatients is
averaged by quarter and plotted in Fig. 2.

Fig. 2. a Proportion of total patients attending a facility who received a clinical diagnosis
of malaria and b proportion of total patients who received AL before training, after training and
after randomization to arms. Results are presented by quarter starting in January
2012 and stratified by transmission zone and study arm

During the 6 months prior to training, about 20 % of all patients in low transmission
facilities were given a clinical diagnosis of malaria and about 15 % of all patients
were given AL. After training, the proportion receiving a clinical diagnosis of malaria
declined by half, but the proportion receiving AL changed only slightly. After randomization
to arms, the proportion of patients receiving AL dropped rapidly in the intervention
arm, but remained stable in the comparison arm for the first 6 months. A similar pattern
is observed in high transmission: before the study, 40 % of patients received a clinical
diagnosis of malaria, but 60 % of all patients received AL. AL prescription increased
slightly in the 3-month period between training and assignment to arms. Both clinical
diagnoses and AL prescription began to decline after the start of the intervention
period.

Overall, the training alone seems to have reduced the proportion of patients assigned
a clinical diagnosis of malaria in low transmission facilities, but did not substantially
change prescription practices in either zone, indicating that AL was being prescribed
to patients with other diagnoses. It is only after the intervention began that substantial
changes in AL prescription are observed.

These numbers show general trends, but it should be noted that they are presented
as raw figures, not adjusted for facility-level covariates. The ratio of clinically
diagnosed patients will be dependent on availability of laboratory services (for example,
some facilities have laboratory services on the weekend while some do not), which
may not be balanced between arms. For the pre-defined study endpoints, we were able
to exclude individual dates when laboratory services were not available, which we
cannot do for aggregated monthly data. Similarly, AL dispensing depends heavily on
the slide-positivity rate and the availability of AL, which also may not be balanced
between arms. Prior to the study initiation, there likely were periodic stockouts
of AL. Finally, routine monthly reporting data is subject to high rates of missingness.