The Applied Statistics Workshop presents another installment this week with
Thomas Cook, Department of Sociology, Northwestern University presenting a
talk entitled, "When the causal estimates from randomized experiments and
non-experiments coincide: Empirical findings from the within-study
comparison literature."
<http://people.fas.harvard.edu/%7Ejgrimmer/Cook.doc>Here is an excerpt
from the paper:
The present paper has several purposes. It seeks to up-date the literature
since Glazerman et al. (2003) and Bloom et al. (2005) and to move it beyond
its near exclusive focus on job training. We have examined the job training
studies, and find nothing to challenge the past conclusions described above.
However, the more recent studies allow us to broach three questions that are
more finely differentiated than whether experiments and non-experiments
produce comparable findings:
1. Do experiments and RDD studies produce comparable effect sizes? We have
found three examples attempting this comparison.
2. Do comparable effect sizes result when the non-experiment depends on
selecting one or more intact comparison groups that are deliberately matched
on pretest measures of the posttest outcome, as recommended in Cook &
Campbell (1979)? Thus, in a non-experiment with schools as the unit of
assignment, intervention schools are carefully matched with intact
non-intervention schools in the hope that the average treatment and
comparison schools will not differ on pretest achievement, let us say,
though they may differ on unobservables. We have found three studies with
this focus.
3. Do experiments and non-experiments produce comparable effect sizes when
the intervention and comparison units do differ at pretest and so
statistical adjustments or individual matches are constructed to control for
this demonstrated non-equivalence? This question has dominated the
literature to date, and we found six studies outside of job training that
asked this question
We will meet at 12 noon in CGIS-Knafel N354 and the talk will begin at
1215pm. And of course a delicious, free lunch will be provided.
Please email me with any questions/concerns/suggestions for the workshop
Justin Grimmer
Show replies by date