Dear Applied Statistics Workshop Community,
Our next meeting of the semester will be on April 12 (12:00 EST). Naoki
Egami will present "Empirical Strategies Toward External Validity:
Framework and External Robustness."
<Where>
CGIS K354
Bagged lunches are available for pick-up at 11:45 (CGIS K354).
Zoom:
https://harvard.zoom.us/j/99181972207?pwd=Ykd3ZzVZRnZCSDZqNVpCSURCNnVvQT09
<Abstract>
Over the last few decades, social scientists have developed and applied a
host of statistical methods to make valid causal inferences, known as the
credibility revolution. This trend has primarily focused on internal
validity — researchers aim to unbiasedly estimate causal effects within a
study. However, one of the most important long-standing methodological
debates is about external validity — how scientists can generalize causal
findings beyond a specific study. This question of external validity has a
long history in the social sciences, going back to at least the 1960s, and
it has recently become even more essential, given that huge opportunities
and challenges of accumulating causal knowledge have become evident.
In this talk, I will discuss a set of empirical strategies to improve
external validity in practice. I briefly introduce a formal framework of
external validity (Egami and Hartman, 2022; APSR) that synthesizes diverse
external validity concerns. Then, I will propose a new simple approach to
quantify the robustness of experimental results to external validity bias
(Devaux and Egami, 2022; Egami and Rothenhäusler, 2023+). In particular, I
introduce a measure of external robustness, which ranges from 0 to 1 and
represents how well causal effects estimated in one’s study can be
generalized to other populations and contexts. Researchers can estimate
this quantity using only experimental data (i.e., no additional data
collection), and users can also account for unmeasured confounders. I
discuss a debiased estimator, which is consistent and asymptotically normal
under mild rate conditions that allow for the use of machine learning
estimators. Finally, I provide default benchmarks and discuss practical
guides about how to report external robustness in practice using the R
package “exr” (
https://github.com/naoki-egami/exr).
Papers: (1)
https://naokiegami.com/paper/external_full.pdf (2)
https://naokiegami.com/paper/external_robust.pdf
<2022-2023 Schedule>
GOV 3009 Website:
https://projects.iq.harvard.edu/applied.stats.workshop-gov3009
Calendar:
https://calendar.google.com/calendar/embed?src=c_3v93pav9fjkkldrbu9snbhned8…
Best,
Shusei