Multivariate meta-analysis refers to the statistical analyses of a collection of studies where at least some studies provide multiple effect-size estimates that may or may not represent multiple constructs. Multiple endpoint studies typically involve research designs where individuals in one treatment group and one control group produce measures on multiple variables. These types of studies will likely lead to statistically dependent effect sizes. The dependence that arises from multiple endpoint studies in the meta-analytic framework has not been thoroughly studied. The main purpose of this thesis was to investigate the impact of dependence from multiple endpoint studies utilizing homogeneity measures commonly found in current meta-analyses.
This thesis is comprised of two sections: simulation and model estimation. Both sections replicated 3,000 meta-analyses for varying conditions. The simulation section varied study sample size, number of studies, between-outcomes correlation, and dependency structure. The model estimation section used generalized least squares estimation to analyze many of the same conditions. The standardized mean difference was the utilized effect-size estimator and Type I error rates of Q statistics were the primary unit of analysis.
Results showed that increased dependence among effect sizes is associated with increased Type I error rates of Q statistics. More specifically, under very dependent conditions, Type I error rates were significantly greater than their nominal levels regardless of within-study sample size and number of studies, sometimes with more than a twofold inflation. The model estimation section demonstrated that using generalized least squares estimation to account for multiple endpoint dependency maintains Type I error rates within nominal levels and is preferable to incorrectly assuming independence.
