Trial design and participants
The PRONTO training intervention was carried out between July and December 2012 at
15 intervention facilities in the four Northern Guatemalan districts of San Marcos,
Alta Verapaz, Quiche, and Huehuetenango. These clinics received PRONTO training as
part of a larger, pair-matched, cluster randomized controlled trial in 30 clinics
with clinical training and community components. This report is limited to an analysis
based on pre/post tests administered only to intervention clinics, as control clinics
received no intervention and provider testing was not feasible at those sites.
Clinics were excluded from the study if they were not open 24-h, had an operating
room, had fewer than an average of seven births per month (in the six months prior
to the intervention), or if they were located more than six hours by vehicle to the
district’s commercial center. All clinics were located in communities with TBAs and
had basic equipment and personnel necessary for vaginal deliveries. In Guatemala,
TBAs have no formal training and in recent years the government has encouraged incorporating
TBAs into clinics as support staff and doulas. The PRONTO team encouraged clinics
to invite community TBAs to participate in the training. Training participants were
chosen by clinic directors from the medical personnel who care for women and babies
during labor and birth or immediately postpartum and their roles in the simulations
were based on their real scope of practice. Providers at the intervention clinics
voluntarily participated in the training and received no compensation other than their
typical salary, transportation and meals. All trainees were informed that they were
in the intervention arm of the trial, and provided verbal informed consent to the
use of their written evaluations for research purposes. The study was reviewed and
approved by the World Health Organization (WHO) Research and Ethics Review committee,
the Independent Latin Ethics Committee, the Guatemalan National Committee for Ethics
in Health, Ministry of Public Health, and the University of Washington IRB, Seattle
(Application # 41922). The trial is registered at the NIH Current Cluster Trial Registration
database (http://www.clinicaltrials.gov; identifier: NCT01653626). The study setting and methods were published elsewhere
23].
Intervention
The PRONTO training intervention utilizes highly realistic yet low-technology, low-cost
simulation materials. Simulation scenarios take place within the actual clinic setting
using only resources that are normally available on-site. PartoPantsâ„¢, the hybrid
simulator worn by a patient actress to simulate obstetric emergencies, were designed
by the PRONTO team 24]. PartoPantsâ„¢ are made from recycled surgical scrub pants and are modified to have
a vaginal opening and pocket to hold an IV bag with tubing for simulated blood. A
cloth doll and placenta, delivered through the PartoPantsâ„¢, are used to simulate the
neonate and placenta during birth. The Laerdal NeoNatalie© is used for neonatal resuscitation
scenarios.
PRONTO consists of two modules conducted two to three months apart that cover topics
of teamwork, communication, intercultural fluency, obstetric hemorrhage, neonatal
resuscitation, preeclampsia/eclampsia, and shoulder dystocia (Fig. 1). Training sessions have minimal formal didactics and are comprised of interactive
team-building exercises, case-based learning, targeted skill sessions, simulation
of obstetric and neonatal emergencies, video-guided debriefings and strategic planning
sessions. Teamwork and communication concepts, adapted from Team STEPPSâ„¢ Training
Program, are taught and reinforced throughout both modules 25], 26]. Evidence-based practices for physiologic birth and respectful, culturally humble
care are also integral components. The curriculum is based upon WHO standards in maternal
and newborn care, and was modified for use in Guatemala to meet the Guatemalan Ministry
of Health guidelines 27]. The training was specifically adapted to address common cultural and language barriers
between the patient population and the facility providers in Northern Guatemala by
addressing cultural humility and humanized birth training in the modules 28]. Finally, during the strategic planning sessions, the provider teams identify actions
aiming for clinical improvement based on weaknesses, they self-identify through the
simulation and debriefs. No intervention took place at the control clinics.
Fig. 1. Timeline and components of PRONTO training intervention in Guatemala
Outcome measures
The primary outcomes included average change in participant self-efficacy and knowledge
scores for each training topic area. Training participants at each intervention clinic
completed pre- and post- training questionnaires for Modules I and II, which evaluated
knowledge of evidence-based practice and confidence in their own ability to perform
key skills (self-efficacy). The questionnaires were adapted from instruments used
in the PRONTO pilot and subsequent hospital-level controlled trial in Mexico. The
Module I pretest and posttest included 58 questions, which assessed knowledge and
self-efficacy of obstetric hemorrhage and neonatal resuscitation, and general obstetric
emergency self-efficacy. The Module II pretest consisted of 36 questions to assess
knowledge and self-efficacy on topics of preeclampsia/eclampsia and shoulder dystocia.
The Module II post-test included 94 topical questions from all of the five topics
covered throughout Module I and II and was administered at the end of the training
intervention. Questionnaires were anonymous and participants were given a study ID
to link the pre-post module answers. Knowledge for each topic area was assessed on
a scale of 0–100 based on the percentage of correct questions answered. Answers left
blank or incorrectly marked with multiple responses were coded as wrong. For each
self-efficacy question, participants were asked to assess their level of confidence
in performing an array of simple to complex practices related to obstetric and neonatal
emergencies using scales of 0–100 based on Bandura’s model of self-efficacy 29], 30]. Scores for each topic area were averaged based on these scales.
Trainees worked in clinic-based teams to decide on goals for improving obstetric/neonatal
care, teamwork, work processes, and infrastructure at their site at the end of Module
I during a facilitated quality improvement planning session. We grouped goals into
four broad categories (teamwork, training, healthcare systems change, and intercultural
fluency). There was no stipulation on the number of goals clinics were asked to select.
During a follow-up session at the beginning of Module II, trainers evaluated their
teams’ achievement of set goals (yes/no). Goal achievement was self-reported, but
when possible, system change goals were confirmed by direct observation by PRONTO
training personnel. Ongoing research is underway to examine the extent of this intervention’s
impact on practice improvement. However, for this phase of the study, clinical improvement
goal achievement was used as a proxy measure of practice change.
Statistical analysis
We used descriptive statistics to characterize participating clinics and trainees.
Knowledge and self-efficacy changes were assessed in the areas of obstetric hemorrhage,
neonatal resuscitation, preeclampsia/eclampsia, and shoulder dystocia. Change in general
obstetric emergency self-efficacy was also assessed. Variables for overall changes
in knowledge/self-efficacy for Module I and for Module II were also created. A combined
variable for overall knowledge change from the start of the training Module I until
the conclusion of training Module II was created for obstetric hemorrhage and neonatal
resuscitation. We estimated the impact of the training on test scores by fitting a
longitudinal fixed-effects linear regression model with fixed effects at the individual
level. The outcome variable was the test score of knowledge or self-efficacy and the
dummy variables for time (pre/post training) served as independent variable. The model
is represented by the equation:
Where Y i,t
is the score for individual i at time t and Y i,0
is the score at baseline, t1
 and t2
are dummy variables for the first and second measurements and ?1
, ?2
are the average differences in score relative to baseline, ? i,t
represents random variability at time t. The fixed-effects approach yields an estimate
of the within-subject change in the outcome variables (knowledge and self-efficacy)
31]. Robust standard errors with clustering were calculated to take into account within-facility
correlation 32]. A second set of models included a time*facility interaction term to test for possible
between-facility heterogeneous effects of the training. To characterize the relationship
between knowledge and self-efficacy score, the correlation between the two scores
was calculated using Pearson’s correlation coefficient for both baseline and follow-up
values. Finally, we used descriptive statistics to describe goal setting and achievement.
As a secondary exploratory analysis, we analyzed clinic-level factors possibly associated
with goal achievement. To do this, we used Poisson regression with number of goals
achieved as the response variable, controlling for number of goals set. Covariates
tested for an association with total number of goals achieved were pre/post Module
I average change in self-efficacy, pre/post Module I average change in knowledge,
training site equipment availability, percent of overall staff attending training,
and average age, sex, and profession of participating providers. These variables were
selected for the plausibility of an association with success in achieving established
goals.
All results are presented as overall and stratified by profession given a previous
and reasonable learning differential for auxiliary nurses versus doctors/professional
nurses. Significance was defined at p-value??0.05, and all analyses were performed
using STATA v. 12.0 33].