Estimands and Estimators for Multi-Site Randomized Trials
Stephen W. Raudenbush University of Chicago
Presented at the Harvard Applied Statistics Seminar September 13, 2017
(Joint work with Daniel Schwartz, University of Chicago)
In a multi-site randomized trial, sites such as schools or hospitals are sampled; within each site, persons are assigned at random to treatments. Such studies are increasingly common in social welfare, medicine, and education. In this talk, I’ll first use potential outcomes and a super- population framework to precisely describe different potential populations and parameters of interest, which may diverge considerably when treatment effects vary. Second, I’ll show that maximizing a weighted two-level likelihood produces consistent estimators of all parameters, but only after we introduce a correction for estimating between-site variance components. Third, we’ll see that these weighted estimators, while consistent, may be embarrassingly inefficient (to the point of being improved by throwing out data). Precision weighting may help but may introduce large-sample bias. In the interest of time, I will focus on two parameters: (1) the average impact of treatment assignment (“intention to treat effect”); (2) in trials with non- compliance, the average impact of participation in the treatment on those induced by random assignment to participate (“complier average causal effect”). I’ll illustrate with data from the National Head Start Impact Study.