The use of a lot quality assurance sampling methodology to assess and manage primary health interventions in conflict-affected West Darfur, Sudan

This paper has described the process of using LQAS to monitor and evaluate the primary health programs of Medair’s operations in West Darfur. Medair used the baseline LQAS findings as an integral part of strategic planning for targeted MNCH interventions and immunization efforts. Based on the results of the MNCH indicators from the first round of LQAS, Medair implemented two interventions to improve maternal and newborn care. First, Medair focused on in-service training for midwives to improve care in the immediate postpartum period for both mothers and infants. The care provided by midwives and supported by Medair was augmented with supervisory visits from advanced-practice midwives, doctors specifically trained in obstetrics and gynecology, and with biannual five to ten day in-service trainings. New strategic topics were introduced during in-service trainings, and the supervisory midwives and doctors visited various clinics to mentor midwives in the implementation of these skills and for quality improvement and quality assurance of the clinical care delivered to mothers and infants in the project area.

Second, Medair trained health promotion volunteers in communities and camps to deliver key messages about the importance of postpartum care. These messages were delivered to groups of mothers in the waiting areas at antenatal clinics and through existing women’s groups (e.g., women’s trade groups). The messages emphasized the importance of delivery with a skilled health worker; the importance of tetanus vaccines, iron supplements, and Vitamin A for the mothers; danger signs for newborns that warrant immediate medical attention; and the importance of a postpartum visit with a midwife or other health worker 2 to 3 days after giving birth and 6 weeks after the birth. Medair used data on key indicators from each round of LQAS to adjust and maximize the interventions after each round. The LQAS-based ME system would periodically give Medair managers empirical feedback, allowing Medair to improve its programs.

Two out of the four MNCH indicators measured by the survey (protection against neonatal tetanus and births attended by a skilled health worker) showed improvement throughout the period that LQAS was used for ME. Medair’s coverage of the two indicators was also greater than the coverage reported in the 2010 Sudan Household Health Survey for West Darfur State (Table 3). Medair’s coverage of protection against neonatal tetanus ranged from 47.2–69.7 %, while the SHHS coverage was 46.5 %. For the percentage of births attended by a skilled health professional, Medair’s coverage ranged from 34.5–52.7 %, compared with the SHHS coverage of 33.4 %. Medair’s coverage of mothers receiving a dose of postpartum vitamin A did not significantly improve. However, Medair’s coverage ranged from 45.5–61.3 % during the survey period and was higher than the 15.3 % coverage reported in the SHHS (Table 4).

The improvement in protection against neonatal tetanus and births attended by a skilled health worker is associated with the Medair managers’ decision to focus on enhancing MNCH programs in West Darfur during the 19 months that LQAS was conducted. However, the improvements were not uniform across the program areas, and Medair faced obstacles in improving MNCH indicators. For example, supervisory midwives and health promotion advisors had limited access to remote communities in SA2 during all assessment rounds due to continued armed conflict and a lack of security. Another program challenge in SAs 4, 6, and 7 was related to two human resource challenges: (1) the number of available trained midwives serving the population was insufficient for the population size, according to Sphere standards [30]; and (2) local labor laws – which necessitated specific work schedules and pay for on-call hours – resulted in interruptions in services.

Throughout the 19-month program implementation period, immunization indicators other than tetanus did not statistically improve, and in contrast to the MNCH indicators, the program coverage in the last assessment period was lower than figures reported in the 2010 SHHS (Table 5). The coverage of the Pentavalent vaccination was 42.1 % during the last assessment period, compared with a coverage of 45.8 % from the SHHS in West Darfur while Medair’s coverage of the measles vaccination was 54.1 % in the last assessment period compared with a coverage of 54.4 % from the SHHS in West Darfur. This may be due to the fact that Medair’s responsibilities were primarily supportive rather than active in West Darfur immunization activities and, although extensive, appear not to have been sufficient. Immunizations were led by the West Darfur State Ministry of Health through routine immunization programs and “mop-up” campaigns, which are campaigns in communities that go house-to-house, often as a result of a confirmed outbreak or in a push to ensure stronger coverage of a specific vaccine (e.g., polio) to immunize children who did not receive vaccines in the initial round. Medair’s supportive interventions included providing salary increases for the Expanded Program on Immunization (EPI) workers; intensifying health promotion about immunizations; providing performance-based incentives for community vaccinators; supporting the logistics for vaccines from the state capital to health facilities throughout the state (e.g., cold chain and storage); and strengthening linkages between antenatal care and immunizations areas within health facilities in the region. Changes were made to the physical layout of some clinics (e.g., placing the EPI office next to the rooms for midwifery care) to facilitate the vaccination of newborns when women came for postpartum visits. However, even with periodic data on the status of the immunization programs from LQAS, Medair’s supportive role did not allow Medair the programmatic control over administration of routine immunizations or the conduct of mop-up campaigns as was the case in other Medair programs.

Medair also faced challenges that were anticipated given working conditions in West Darfur. Supervisory Areas 6 and 7 were absorbed into Medair’s existing programs when other international NGOs were expelled in the first quarter of 2009. Medair started programs in these communities in late 2009. In the interim period, the West Darfur Ministry of Health worked to maintain the health facilities in this area, but a number of health workers resigned and several health facilities closed for a period of time. Medair’s program implementation included reopening these facilities, filling vacant health worker positions, initiating health promotion activities in the communities, and ensuring that clinical care met standards set by both Medair and the State Ministry of Health. There were also logistical challenges in the area, particularly in the rainy season, when some of the health facilities and trade centers were inaccessible due to flooding.

With these challenges, the data reflect poor performance in SAs 6 and 7. SA6 was classified as inadequate for the receipt of clean delivery kits in three of the four assessment periods, including the final round of LQAS. It was also classified as inadequate for two of the three immunization indicators in the final round. Similarly, SA7 was classified as inadequate for the coverage of measles immunization in every LQAS round and for two of the three immunization indicators in the December 2012 assessment.

Active armed conflict and the lack of security also hampered the efforts of Medair to deliver services to the catchment area and to assess the effectiveness of these services. Medair planned monthly quality assurance, supervision, and mentoring visits for clinical staff and health education volunteers, but frequently these visits were delayed due to security restrictions. When on-site visits were possible, Medair staff had limited time for these visits with local clinic staff and volunteers. At times, field visits of 3–4 h each quarter were the only times allowed for these interactions, compounding the challenges of conducting LQAS. Thus, while Medair was able to feasibly train and conduct the LQAS ME program in a conflict setting, LQAS was a necessary but not sufficient element in translating the ME system into improvements for Medair’s beneficiaries. LQAS by itself cannot overcome the barriers to healthcare access created by conflict and large distances between towns. If such barriers are eliminated, however, LQAS helps managers to efficiently monitor and adjust their programs.

There are limitations to the data we present. As mentioned above, accurate population estimates in Darfur are difficult to obtain; it is possible that our population estimates were inaccurate, which would affect the formulation of the sampling plan and introduce bias into our results. We did not record the number of individuals who declined to participate in the survey. Women who did participate may be systematically different than those who declined, which may also bias our results. Furthermore, during our first assessment, Medair staff was unable to enter SA2 due to armed conflict. The absence of this data in addition to the small sample sizes in LQAS in general decreases the precision of the estimates of indicators and makes drawing conclusions more difficult. As already described, access to clinics was hampered during the rainy season, and differences in indicators from one assessment period to the next six-month assessment round may have been affected by these seasonal factors. While there are other types of complex humanitarian emergencies for which LQAS may be optimal, we only examined the use of LQAS in a conflict-affected setting. Our conclusions may not apply in other types of humanitarian emergencies.

Another limitation of the data is that Medair restricted its use of LQAS to assess the population coverage of specific services related to the organization’s programs. The monitoring and evaluation system for this project focused on coverage, but did not include measurements of quality of care. Medair utilized other quality assurance measures in their supportive supervision of clinics and community health promotion volunteers. LQAS was specifically utilized to help determine the coverage of specific interventions and education initiatives that impact key health issues. Additionally, Medair needed to limit the length and time for the survey questionnaire and the limited available time of surveyors in communities. However, LQAS can be used as a tool to assess quality of care. There are multiple examples of LQAS being used for quality management [15], such as assessing the quality of care in malaria control in Nigeria [19], and the quality of health facility services in South Sudan [22]. While quality of care was not specifically measured in the presented study in West Darfur, we recommend using LQAS for this purpose in the future so that the full range of the program’s expected results (coverage and quality) can be monitored on a regular basis in a conflict-affected environment.

The next steps from the findings we present are to implement LQAS in a different complex humanitarian emergency to examine if our experience using LQAS in Darfur is consistent in other contexts. While there are published papers describing the combination of LQAS with aspects of the cluster survey design [21, 31], further investigations should also focus on comparing LQAS with the traditional cluster survey method in a complex emergency, with an emphasis on comparing the feasibility of the two survey methods to understand more clearly which method is preferable in various settings. In addition, applications of LQAS as a quality management tool in humanitarian emergencies should be further tested, including more robust efforts to support use of periodic or mid-program data to redirect activities as issues are identified. Until such studies are conducted, this paper has taken a step in illustrating how a novel method of quality assurance was adapted to provide a tool for management of a health program in an insecure, dynamic environment.