Abstract: Text data have a long history in social science and education research. However, these data are notoriously high-dimensional and characterized by many nuances of language that lack plausible statistical models. As a result, analysis of text data typically involves intensive human coding tasks where particular constructs or features of the text are first defined, and then a collection of documents are inspected and coded for the presence or absence of these constructs. While this process may be feasible in studies with smaller sample sizes, the time and resources required to train and employ multiple human coders frequently poses a challenge for large-scale efforts. In this talk, I will consider how to reliably and efficiently extract meaningful constructs from text documents for the purposes of drawing causal inferences, with an emphasis on the context of experimental studies where some outcomes of interest are features of text generated by the trial’s participants. In particular, I will describe an approach that combines machine learning and survey sampling methods to streamline the process of hand-coding in a way that is automatically verified and validated. To illustrate the proposed methods, I will present results from a pilot analysis of a randomized trial that used student-generated essays to evaluate the impact of an educational intervention on students’ writing abilities.
Where: CGIS Knafel Building, Room K354 (see this link for directions).
When: Wednesday, March 11 at 12noon - 1:30pm.
All are welcome and lunch will be provided.
Best,
Georgie