Wednesday, February 6, 2019
CGIS Knafel Building (K354) - 12-1:30 pm
Abstract: We propose sensitivity analyses for selection in meta-analysis due to publication bias, selective reporting, and "p-hacking". We consider a publication process such that "statistically significant'' positive results are more likely to be published than negative or "nonsignificant'' results by an unknown ratio. Using inverse-probability weighting and robust estimation that accommodates non-normal true effects, small meta-analyses, and clustering, we develop sensitivity analyses that enable statements such as: "For publication bias to shift the observed point estimate to the null, 'significant' positive results would need to be at least 30-fold more likely to be published than negative or 'nonsignificant' results.'' Comparable statements can be made regarding shifting to a chosen non-null value or shifting the confidence interval. We show that a worst-case meta-analytic point estimate under maximal publication bias can be obtained simply by conducting a standard meta-analysis of only the negative and "nonsignificant'' studies; this method sometimes indicates that no amount of publication bias could "explain away'' the results. We illustrate the proposed methods using real-life meta-analyses. An R package is forthcoming.
Maya Mathur is a postdoctoral fellow in the Department of Epidemiology at Harvard University.