[gov3009-l] Applied Statistics Workshop: Luke Miratrix on Wed., Sept. 26

estimates of treatment effects using baseline covariates thought to predict the outcome of interest. In a previous paper, we proved that even under the Neyman-Rubin model, if the covariates and the method for adjustment are determined before randomization, this process can increase precision in a manner quite similar to a comparable blocked experiment. Typically, however, experimenters wish to adjust for the covariates that are most imbalanced between treatment and control, given the realized randomization. This leads to a much vexed variable selection problem that depends on the observed treatment assignment. To understand the issues behind this process, we examine a class of estimators we call "Random Weight Estimators" that adjust treatment effect estimates by weighting units with weights depending on a function on treatment assignment and covariates. While similar in spirit to blocking, these estimators can be applied "after the fact,'' i.e., after randomization has occurred, allowing them to naturally adapt to the observed treatment assignment. They can also adjust for many different covariates at once, including continuous ones. This class is quite general, and it includes traditional methods such as ordinary linear regression. Using our framework, we show, under the Neyman-Rubin model, how one can easily introduce potential bias using what would seem to be legitimate and simple approaches, especially in small and midsize experiments. Care must be taken with many forms of adjustment, even if an approach is selected without regard to any actual outcomes. We also extend this methodology to survey experiments, giving an appropriate and near-unbiased estimator for the treatment effect of a parent population. Throughout the talk, we illustrate this overall framework.