Dear Applied Statistics Workshop Community,

Our next meeting of the semester will be on April 12 (12:00 EST). Naoki Egami will present "Empirical Strategies Toward External Validity: Framework and External Robustness."

<Where>
CGIS K354
Bagged lunches are available for pick-up at 11:45 (CGIS K354).
Zoom: https://harvard.zoom.us/j/99181972207?pwd=Ykd3ZzVZRnZCSDZqNVpCSURCNnVvQT09

<Abstract>
Over the last few decades, social scientists have developed and applied a host of statistical methods to make valid causal inferences, known as the credibility revolution. This trend has primarily focused on internal validity — researchers aim to unbiasedly estimate causal effects within a study. However, one of the most important long-standing methodological debates is about external validity — how scientists can generalize causal findings beyond a specific study. This question of external validity has a long history in the social sciences, going back to at least the 1960s, and it has recently become even more essential, given that huge opportunities and challenges of accumulating causal knowledge have become evident.

In this talk, I will discuss a set of empirical strategies to improve external validity in practice. I briefly introduce a formal framework of external validity (Egami and Hartman, 2022; APSR) that synthesizes diverse external validity concerns. Then, I will propose a new simple approach to quantify the robustness of experimental results to external validity bias (Devaux and Egami, 2022; Egami and Rothenhäusler, 2023+). In particular, I introduce a measure of external robustness, which ranges from 0 to 1 and represents how well causal effects estimated in one’s study can be generalized to other populations and contexts. Researchers can estimate this quantity using only experimental data (i.e., no additional data collection), and users can also account for unmeasured confounders. I discuss a debiased estimator, which is consistent and asymptotically normal under mild rate conditions that allow for the use of machine learning estimators. Finally, I provide default benchmarks and discuss practical guides about how to report external robustness in practice using the R package “exr” (https://github.com/naoki-egami/exr).