The Functionality Assessment Flowchart (FAF): a new simple and reliable method to measure performance status with a high percentage of agreement between observers

Cancer treatments are initiated and terminated based on PS scores; inaccurate estimates
may lead to a failure to receive treatment that may be helpful or to a patient receiving
an aggressive treatment that should have been avoided. Moreover, the PS is largely
used to select participants for inclusion in clinical trials. Thus, PS assessment
is an essential part of oncological care and must be evaluated with high accuracy
levels. In the present study, we present a simple and reliable flowchart that considers
patient opinions and that demonstrates high absolute concordance rates and good construct
validity.

The FAF is a new method to evaluate the PS of patients with cancer, compensating for
the lack of instruments to measure functionality in detail (on an 11-point scale)
with a high concordance rate between observers. The absolute concordance rate in the
present study yielded nearly 80 % agreement, which was much higher than the absolute
agreement of the KPS (~50 %) and ECOG-PS (67 %). Regarding the ECOG-PS, previous studies
found absolute agreement ranging from 40 % to 93 % 1], 9], 14], 15]. The inter-observer variability increases as the number of choice increases 6]. Thus, the absolute agreement rate of the KPS between observers is generally lower
than that of ECOG-PS, varying from 38 % to 76 % 1], 2], 9], 15].

Previous studies evaluated the agreement rates between observers by performing correlation
analyses. In general, high correlation coefficients (r??0.80) have been observed for ECOG-PS and KPS 2], 9], 16]. In accordance with previous studies, we found Spearman correlation coefficients
of approximately 0.9 for all three of the evaluated scales. Moreover, our study highlights
that high correlation levels are not necessarily associated with high agreement between
raters.

Although the overall percentage of agreement provides a measure of agreement, it does
not consider the agreement that would be expected purely by chance. The kappa statistic,
however, is a measure of “true” agreement 17]. We found a clearly higher value of the kappa statistic for FAF compared with that
for KPS. However, considering that our instruments are all ordinal multi-category
scales, kappa can be weighted to confer greater importance to large differences than
small differences between ratings. The KPS and FAF weighted kappa values were similar,
suggesting that the disagreements between observers regarding KPS were primary small
differences. The same pattern of improvement in agreement values from unweighted to
weighted kappa were also observed by Meyers et al. 9].

One advantage of the FAF over the other tested scales is that it considers the patient’s
opinion about their own functional states. As we hypothesized, the FAF can improve
the concordance rates between raters. However, some women could have inaccurately
answered the first step of the FAF (“Are you able to work or to do your daily activities?”),
causing secondary gains by considering themselves worse (leave or absence from work
due to illness) or better (as a way to feel more optimistic) than they actually were.
FAF raters must understand that the FAF is a flowchart developed to facilitate PS
evaluation and not a rigid measure based strictly on patient responses.

The lack of a functional gold standard tool was a challenge for this study. Thus,
to evaluate the construct validity of the FAF, we compared its scores with functional
and fatigue scores obtained from the previously validated Brazilian version of the
FACT-F questionnaire 11]. As expected, the correlation between the functional and fatigue scores and the PS
scales was strong. Therefore, in terms of construct validity, the FAF should be considered
as valid as ECOG-PS and KPS.

Study limitations

This study was preliminary; therefore, one limitation was its small sample size. Another
significant limitation is that all of the study assessments were performed repeatedly
at the same ambulatory setting. Only female participants were included, which potentially
reduces the generalizability of our results. Although we analyzed many low-functioning
participants selected from the waiting rooms, future studies should include a greater
sample of both outpatients and inpatients.

Future perspectives

Our preliminary findings support a subsequent study with a larger and heterogeneous
sample size to more definitively investigate the benefit of implementing a PS assessment
using the FAF in clinical practice. We are currently developing a computational software
containing the FAF and intend to assess its construct validity by comparing its values
with more precise functional activity levels measured by digital accelerometers 18]. We consider both the ECOG-PS and KPS to be well-established tools in the oncology
setting. However, the FAF has the advantage of evaluating the PS in a more discriminative
manner than the ECOG-PS and with a higher concordance rate than KPS. Thus, the FAF
is a new tool that requires further refinement and investigation.