Analysis of multivariate longitudinal kidney function outcomes using generalized linear mixed models

To assess the performance of each model a pseudo simulation bootstrap study was conducted
using a renal dataset and assuming that the three kidney outcomes (BUN, Cr and eGFR)
follow the lognormal distribution. The covariates were limited to time and square
of time since SPRIS exhibited convergence problems when other covariates were included.
The square of time was included in the analysis to account for the curvilinear relationship
between time and each of the outcomes. We implemented 2,000 replications each with
a sample size of 110 individuals boot strapped from the cohort of renal patients.
The performance of the 4 multivariate models (SHRI, SHRIS, SPRI, SPRIS) and that of
the univariate random intercept model (RI), and random intercept and slope model (RIS)
was examined. The performance was determined in terms of accuracy assessed by two
measures; the mean of the differences between the estimates of the bootstrapped datasets
and the true beta obtained from the application of each model on the renal dataset,
and the mean of the square of these differences. Specifically, true betas are the
fixed effects in the models’ equations. These parameters are estimated by applying the generalized
linear mixed models on the renal transplant dataset to reflect the estimated effects
of the corresponding covariates on the changes in kidney outcomes. In Table 1, the differences between the estimates generated in the simulation studies and these
betas are computed to determine the accuracy of the various models, noting that the
smaller the difference the more accurate the model. For simplicity, we refer to these
measures as mean difference, and mean squared difference respectively. This comparison
helped us achieve the following objectives:

1. Identify the multivariate models that exhibited better performance than the other
models.

2. Determine whether the complex separate random models (SPRI and SPRIS) had any advantage
in terms of accuracy of the estimates over the simpler shared random effect models
(SHRI and SHRIS). Accordingly, we can gauge complexity versus precision.

3. Examine whether including the slope as random effect along with the intercept as
in the SHRIS and SPRIS models has any respective advantage over the models that assume
solely random intercept as in SHRI and SPRI.

4. Evaluate whether there was a gain in accuracy under the multivariate modeling versus
the univariate separate analysis implemented on every outcome.

Table 1. Bootstrapping on the (a) multivariate models and (b) univariate models: replicate
2,000 sample size 110

The bootstrap results are presented in Tables 1, 2, and 3 for the multivariate and univariate implementation of the models with 2,000 replications.
In Table 1 we present the mean difference and mean squared difference for the estimates under
every model while in Table 2 we provide percent reductions in these measures for a specific multivariate model
versus the others. In Table 3 the percent reductions in these measures between univariate and multivariate models
are presented.

Table 2. Percent reduction in mean difference and mean squared difference that correspond to:
SPRI compared to all other multivariate models (a, b, and c)^b; SPRI compared to SHRI (d); SPRIS compared to SHRIS (e); SPRI compared to SPRIS(f)

Table 3. Percent reduction in mean difference that correspond to: SPRI Compared to RI (a);
SPRIS compared to RIS (b)

Objective 1 As evidenced in Tables 1 and 2 (see also 2nd footnote labeled as b in Table 2, and Table 2 columns a, b, and c), we notice that 6 out of 12 estimates i.e. 50% of the estimates
generated under SPRI had the smallest mean difference, and mean squared difference
compared to those generated under the other models that include SHRI, SHRIS, and SPRIS.
Specifically, 50% of the estimates generated under SPRI had an associated mean difference,
and mean squared difference that are respectively 58.7% and 12% smaller on average
than those generated under the other models. Hence, this leads to the conclusion that
SPRI demonstrated the best performance compared to the other 3 models.

Objective 2 Compared to SHRI, 83% of the estimates generated under SPRI had associated mean difference
that is 40% smaller on average, and mean squared difference that are close in value
(15% reduction in mean squared differences under SPRI in 50% of the estimates) (see
Table 2 column d, and footnote Table 2). In this context, we denote that the eGFR outcome had a significant 32% reduction
in mean squared difference under SPRI compared to SHRI. When we compared SPRIS to
SHRIS (see Table 2 column e, and footnote Table 2), we realized that about 50% of the estimates generated under SPRIS exhibited an
approximate average reduction of 73% in mean difference compared to SHRIS; meanwhile,
an observed reduction of 19% in mean squared difference was captured in only 33% of
the estimates under SPRIS. Hence, mean squared difference did not greatly differ between
SHRIS and SPRIS. Accordingly, one can conclude that separate modeling of the random
effect had it be solely intercept (SPRI), or both intercept and slope (SPRIS) had
an advantage specifically in mean difference reduction over the simpler models that
assume shared random effects (SHRI and SHRIS).

Objective 3 To assess whether the increase in complexity when going from the simpler model (SPRI)
to the more complex one (SPRIS) generates more accurate results, we observed that
66% of the estimates under SPRI had an associated mean difference that is 72% smaller
on average than those generated under SPRIS, and 100% of the estimates under SPRI
had smaller mean squared difference (9% average reduction in mean squared difference)
than those generated under SPRIS (see Table 2 column f, and footnote Table 2). Hence, the simpler model (SPRI) appeared to generate more accurate estimates than
the complex model (SPRIS). This conclusion supports the finding established in objective
1. However, this was not the case with SHRI and SHRIS. Specifically, when the performance
of the simpler model (SHRI) was compared to that of the more complex one (SHRIS),
similar overall performance was observed for both models wherein both models had close
mean difference and mean squared difference (see Table 1 SHRI and SHRIS models).

Hence SPRI appeared to have better performance than the other models and increasing
the level of complexity by going from SPRI to SPRIS did not improve on the accuracy
of the estimates that were generated under SPRI. Hence, these results lead to the
conclusion that modeling the slope as a random effect along with the intercept did
not improve the performance of the associated models (SHRIS and SPRIS) compared to
the simpler models that assume solely random intercept (SHRI and SPRI).

Objective 4 Our bootstrap pseudo simulation study also aimed at investigating the effect of multivariate
modeling versus the univariate implementation of the models that assume that the outcomes
are independent. In this regard a comparison was undertaken between the performance
of the RI model and that of SHRI and SPRI, and between RIS and that of SHRIS and SPRIS
(Tables 1, 3). The performance of SHRI and RI did not significantly differ since their associated
mean difference and mean squared differences were close (see Table 1). Hence, this observation indicates that under models with random intercept, the
multivariate shared model (SHRI) did not improve the accuracy of the estimates compared
to those generated under the univariate random intercept model (RI). However, this
conclusion did not hold when we compared SPRI and RI. Specifically, we noticed that
66% of the estimates generated under SPRI had a 42% average reduction in mean difference
compared to those estimates generated under RI (see Table 3 column a, and footnote Table 3). For instance, 77 and 73% reductions in mean difference were observed for the slope
estimates of the outcomes creatinine and eGFR generated under SPRI compared to those
generated under RI. Nonetheless, the mean squared differences did not greatly differ
in the estimates of these two models.

When SHRIS and RIS were compared, a minimal reduction (only 4%) was denoted for the
mean squared differences in the majority of the estimates (about 80%) that were generated
under SHRIS compared to RIS; but the mean differences did not differ between the 2
models. Hence SHRIS and RIS exhibited similar performance. Compared to RIS, about
50% of the estimates generated under SPRIS exhibited an observed reduction in the
mean difference by about 41% on average (see Table 3 column b, and footnote Table 3), while the mean squared difference did not greatly differ between the 2 models.
These results suggest that multivariate modeling specifically SPRI and SPRIS had an
advantage in performance specifically in Mean Difference compared to the univariate
models RI and RIS respectively (Table 3). However, this conclusion did not hold for the shared multivariate models SHRI and
SHRIS wherein their associated mean difference and mean squared difference did not
greatly differ from those under the univariate RI and RIS respectively. Hence only
the separate multivariate models (SPRI and SPRIS) exhibited improved performance over
their univariate counterpart models RI and RIS. This result also endorses our conclusion
in objective 2 which stated that separate random effect models (SPRI and SPRIS) had
better performance compared to the shared random effect models (SHRI and SHRIS).