You are here

UCF Digital Collections ยป ETD Collection

Pooling correlation matrices corrected for selection bias: Implications for meta-analysis

Download pdf | Full Screen View

Date Issued:
2019
Abstract/Description:
Selection effects systematically attenuate correlations and must be considered when performing meta-analyses. No research domain is immune to selection effects, evident whenever self-selection or attrition take place. In educational research, selection effects are unavoidable in studies of postsecondary admissions, placement testing, or teacher selection. While methods to correct for selection bias are well documented for univariate meta-analyses, they have gone unexamined in multivariate meta-analyses, which synthesize more than one correlation from each study (i.e., a correlation matrix). Multivariate meta-analyses of correlations provide opportunities to explore complex relationships and correcting for selection effects improves the summary effect estimates. I used Monte Carlo simulations to test two methods of correcting selection effects and evaluate a method for pooling the corrected matrices. First, I examined the performance of Thorndike's corrections (for both explicit and incidental selection) and Lawley's multivariate correction for selection on correlation matrices when explicit selection takes place on a single variable. Simulation conditions included a wide range of selection ratios, samples sizes, and population correlations. The results indicated that univariate and multivariate correction methods perform equivalently. I provide practical guidelines for choosing between the two methods. In a second Monte Carlo simulation, I examined the confidence interval coverage rates of a Robust Variance Estimation (RVE) procedure when it is used to pool correlation matrices corrected for selection effects under a random-effects model. The RVE procedure empirically estimates the standard errors of the corrected correlations and has the advantage of having no distributional assumptions. Simulation conditions included tau-squared ratio, within-study sample size, number of studies, and selection ratio. The results were mixed, with RVE performing well under higher selection ratios and larger unrestricted sample sizes. RVE performed consistently across values of tau-squared. I recommend applications of the results, especially for educational research, and opportunities for future research.
Title: Pooling correlation matrices corrected for selection bias: Implications for meta-analysis.
59 views
38 downloads
Name(s): Matthews, Kenneth, Author
Sivo, Stephen, Committee Chair
Bai, Haiyan, Committee Member
Hahs-Vaughn, Debbie, Committee Member
Butler, Malcolm, Committee Member
University of Central Florida, Degree Grantor
Type of Resource: text
Date Issued: 2019
Publisher: University of Central Florida
Language(s): English
Abstract/Description: Selection effects systematically attenuate correlations and must be considered when performing meta-analyses. No research domain is immune to selection effects, evident whenever self-selection or attrition take place. In educational research, selection effects are unavoidable in studies of postsecondary admissions, placement testing, or teacher selection. While methods to correct for selection bias are well documented for univariate meta-analyses, they have gone unexamined in multivariate meta-analyses, which synthesize more than one correlation from each study (i.e., a correlation matrix). Multivariate meta-analyses of correlations provide opportunities to explore complex relationships and correcting for selection effects improves the summary effect estimates. I used Monte Carlo simulations to test two methods of correcting selection effects and evaluate a method for pooling the corrected matrices. First, I examined the performance of Thorndike's corrections (for both explicit and incidental selection) and Lawley's multivariate correction for selection on correlation matrices when explicit selection takes place on a single variable. Simulation conditions included a wide range of selection ratios, samples sizes, and population correlations. The results indicated that univariate and multivariate correction methods perform equivalently. I provide practical guidelines for choosing between the two methods. In a second Monte Carlo simulation, I examined the confidence interval coverage rates of a Robust Variance Estimation (RVE) procedure when it is used to pool correlation matrices corrected for selection effects under a random-effects model. The RVE procedure empirically estimates the standard errors of the corrected correlations and has the advantage of having no distributional assumptions. Simulation conditions included tau-squared ratio, within-study sample size, number of studies, and selection ratio. The results were mixed, with RVE performing well under higher selection ratios and larger unrestricted sample sizes. RVE performed consistently across values of tau-squared. I recommend applications of the results, especially for educational research, and opportunities for future research.
Identifier: CFE0007680 (IID), ucf:52483 (fedora)
Note(s): 2019-08-01
Ph.D.
Community Innovation and Education, School of Teacher Education
Doctoral
This record was generated from author submitted information.
Subject(s): meta-analysis -- research synthesis -- quantitative methods -- range restriction
Persistent Link to This Record: http://purl.flvc.org/ucf/fd/CFE0007680
Restrictions on Access: campus 2022-08-15
Host Institution: UCF

In Collections