Implementation of a stepped-wedge cluster randomized design in routine public health practice: design and application for a tuberculosis (TB) household contact study in a high burden area of Lima, Peru
This pragmatic stepped-wedge CRCT protocol is among the very few undertaken in an
operational public health program in a LMIC 38]. The following discussion highlights the planning stages and development of this
collaborative study protocol, the initiation and implementation of this study, including
the strengths and challenges of using the stepped-wedge design in the implementation
of an intervention within a public health program under real-world conditions (Table 2).
Table 2. Key advantages and challenges related to public health implementation in a stepped-wedge
pragmatic CRCT
Planning and development of the pragmatic stepped-wedge CRCT protocol
This protocol is a joint endeavor between local NTP partners including the operational
managers of the Ministry of Health TB program in SJL and researchers. Indeed such
collaborations at the outset of operational studies, like the stepped-wedge CRCT in
SJL, are crucial to their success. During the planning stages, a partnership of NTP
programmers with local and international researchers was formed with the common goal
of estimating the added benefit of actively evaluating HCs of TB cases for disease
compared to the existing DOTS TB strategy of symptomatic individuals self-reporting
for evaluation (standard of care). NTP programmers highlighted the importance of having
a rapid intervention rollout in order to meet annual programmatic targets and to demonstrate
public health action across all health centres in their jurisdiction. Typically in
many RCT designs, only half of the included study health centers would be assigned
to the intervention arm 8]. Researchers emphasized the need to plan study design aspects such as random assignment,
sufficient study sample size in intervention or control arms and supervision of health
workers to ensure high quality implementation and sound data management. The stepped-wedge
design most aligned with stakeholder needs, including the introduction of randomized
crossover times and allowing NTP programmers to oversee a controlled implementation
of the intervention across all health centres 26]. This novel design is considered particularly useful in the implementation of public
health interventions, given the phased implementation steps. However stepped-wedge
designs are relatively new, and many programmatic managers may be unfamiliar with
their ability to yield methodologically valid and informative results. Finally, an
upfront understanding of the utility of this implementation design is needed so findings
of this study would considered by local decision makers hearing of the stepped-wedge
CRCT design for the first time.
Researchers can advise on methodological design issues in order to meet the needs
of the research question, however in order to understand the operational research
aspects of the protocol, gaining a thorough understanding of the local daily operations
of the TB program at the health centre level is a key aspect to the design of this
protocol. This knowledge included understanding of how contact investigations are
undertaken routinely, how the intervention will be applied in local NTP centres, and
how monitoring, supervision and data collection could be undertaken feasibly throughout
the study period.
Strengths and challenges of the stepped-wedge design
The stepped-wedge design is particularly useful in this study due to the required
widescale roll-out of this public health intervention. In SJL, the systematic stepped-wedge
implementation of a provider-initiated ACF program amongst HCs of TB cases provides
new evidence, where previously very limited pre-existing effectiveness data are available
and few, if any, have been measured within the local public health program in a TB
endemic area.
While the staggered intervention initiation times increase the flexibility of the
stepped-wedge design in practice, it also requires understanding of the methodological
complexities involved. A stepped-wedge allows for a degree of flexibility in its design,
such as the number of centres at initiation, and enough time between wedges to provide
training and achieve enough study power. However, this design is still subject to
biases and threats to sample size and study power. Determining the number of clusters,
the number of steps and time-frame for rollout must all be predetermined and require
special consideration. If the composition of clusters to be randomized at each crossover
time point is unbalanced, this could result in skewed increases or decreases in the
measured outcomes. For example, if all high TB rate centres were to cross over at
the first time point and all low TB rate centres cross over at the last time point,
this could result in a skewed numbers of secondary TB outcomes in HCs observed earlier
and for a longer time period of the study. A simple stratification for the randomization
of health centres by TB rate at each crossover time point is used in the current protocol
to account for TB burden and as a proxy for the corresponding size of clinics to evenly
distribute centres across the various wedges 30], 31].
A common challenge for many operational research designed studies is that blinding
of the intervention assignment is not possible. In the current protocol, health centres
awaiting crossover may have anticipated their likely crossover time based on the number
of health centres that have already initiated the intervention. Additionally, the
public health intervention is undertaken by health centre staff, who along with patients,
know whether or not they have initiated the intervention. In our study, contamination
between clusters, though possible, is not a major concern as TB patients and their
HCs must attend the health centre of the catchment area to which their primary residence
is registered.
The selection of a stepped-wedge design is considered optimal in the context of logistical,
feasibility and resource challenges. However, the stepped-wedge design of a population-based
implementation does not necessarily require fewer research or programmatic resources
to undertake adequately compared to other study designs. In the current protocol,
given a study period spanning 20Â months, resources were required for monitoring the
full implementation of the intervention across all centers for the entire time frame.
These include study resources for data quality monitoring, training and supervision
of the intervention, and data extraction from health centre charts throughout the
full study period. While all operational research studies are subject to these challenges,
it is a consideration in the implementation of a stepped-wedge design study where
intervention cross over times occurs over a long period. If certain centres or the
entire district interrupt the intervention in health centres for unforeseen circumstances,
such as personnel strikes, public health outbreaks or other political challenges,
then data collection, quality and completeness could suffer and at minimum require
adjustment in the study time lines and addition of resources to complete data collection
or worse, could lead to imbalances or biases in the overall stepped-wedge designed
study.
Strengths and challenges to the stepped-wedge design during implementation within
a routine TB program
There are several advantages and disadvantages to designing a pragmatic stepped-wedge
CRCT for operational research within a routine public health program. A unique feature
of the current protocol is the use of routine NTP program staff. The stepped-wedge
approach using NTP personnel provides an evaluation of intervention effectiveness
within actual programmatic conditions and therefore, may be more representatives then
a study using strictly dedicated highly trained research personnel. The ability to
sustainably integrate the intervention into the program during the study period was
facilitated given the intervention was presented as a function of the routine program
and not a study specific responsibility. The stepped-wedge implementation provides
a useful tool to be able to logistically implement the study, allowing for targeted
training within small groups as they initiate the intervention.
Public health interventions are complex and context dependent, as they are integrated
within the infrastructure of existing health systems, dependent on local political,
socioeconomic and cultural perspectives of the population and its public health practitioners
6]. The use of routine NTP nurses and physicians adds a complexity given that this intervention
of household contact tracing is conducted within the context of numerous other responsibilities
of the health centre staff. TB programs are impacted by complex treatment management
protocols for active TB cases, a high burden of cases in endemic areas and numerous
administrative programmatic activities (e.g., reporting forms, indicators, record
keeping). Therefore, effective implementation requires planning, high quality training
and active monitoring. These factors are better achieved when programs are initiated
in a manageable number of sites; the stepped-wedge design allows for flexibility in
determining the number of sites initiating a new program at any one time. This aspect
is particularly valuable when beginning the new program in all or half of all sites
would not be feasible 39]. The randomized allocation of centres to the crossover time point also eliminates
preferential assignments due to either health centres’ performance evaluations or
other subjective criteria. In the context of the current study, health centre NTP
staff appeared motivated and engaged in training and preparation for the intervention
once it is known that all centres will have to implement the new program and undergo
the same intensive training, monitoring and evaluation processes, instead of some
targeted added work assigned only to half of the centres.
Adherence or fidelity to the program and whether the intervention, in its intended
form, is systematically applied by staff within a health centre is a distinct challenge
particularly in pragmatic trials where routine program staff are undertaking the new
intervention as part of their normal duties 6]. From an analytic perspective, intention-to-treat analysis (ITT) is generally the
preferred analytic strategy for CRCTs, which considers outcomes based on the random
allocation to the intervention arm, regardless of what happened subsequent to assignment.
In practice, if there is no reported difference in effectiveness between intervention
and control arms, conclusions need to consider whether the ineffectiveness of the
intervention is the cause or if possible low adherence of implementers to the intervention
is the more likely explanation. To examine this latter possibility, per protocol analyses,
which consider actual adherence to the intervention, will also be examined in our
study.
Several challenges can occur in undertaking a research study within the daily operations
of a public health program, in addition to high staff turnover, worker strikes, outbreaks
in other disease areas, and authorization requirements for data access. These substantial
issues lead to interruptions of activities, which may not be encountered in studies
using dedicated research staff. While planning and design of the stepped-wedge can
help to control some of these threats, in some instances these cannot be completely
avoided or controlled. The probability of an interruption occurring increase and should
be expected when study periods are projected over several months within a routine
operational TB program. However, dealing with unforeseen events is a common reality
for most public health settings, and the evaluation of the intervention within a pragmatic
setting could reflect its likely effectiveness once integrated into practice.
Concluding remarks
Stepped-wedge designs provide an important option for public health researchers and
practitioners to generate intervention effectiveness data that otherwise could remain
unmeasured. Typically, stepped-wedge designs are justified when feasibility, logistics
and/or limited resources are important practical considerations while still allowing
for a randomization process as part of the operational research. Overall, there is
no indication that a stepped-wedge design requires fewer resources than other designs;
the design requires resources over a longer period of time, yet involves smaller resources
for training and monitoring at any given time point during the study. Finally, fidelity
or adherence to the intervention may need to be considered during implementation and
in the analysis, in order to correctly interpret null or negligible findings of effectiveness.
The current ongoing study will provide invaluable evidence on contextual factors that
would not have been possible in traditional study designs. The findings of this study
will have implications for the selection of interventions and allocation of resources
in TB programming for Peru, and will be a major contribution in the field of TB prevention
and TB contact tracing in LMICs.