Dear Applied Statistics Workshop Community,
Our last meeting of this semester will be on Wednesday, May 1 (12:00 EST).
Kosuke Imai presents "Does AI help humans make better decisions? A
methodological framework for experimental evaluation" (joint work with Eli
Ben-Michael, D. James Greiner, Melody Huang, Zhichao Jiang, Sooahn Shin).
<When>
May 1, 12:00 to 1:30 PM, EST
Lunch will be available for pick-up inside CGIS K354.
<Where>
In-person: CGIS K354
Zoom:
https://harvard.zoom.us/j/93217566507?pwd=elBwYjRJcWhlVE5teE1VNDZoUXdjQT09
<Abstract>
The use of Artificial Intelligence (AI) based on data-driven algorithms has
become ubiquitous in today's society. Yet, in many cases and especially
when stakes are high, humans still make final decisions. The critical
question, therefore, is whether AI helps humans make better decisions as
compared to a human-alone or AI-alone system. We introduce a new
methodological framework that can be used to answer experimentally this
question with no additional assumptions. We measure a decision maker's
ability to make correct decisions using standard classification metrics
based on the baseline potential outcome. We consider a single-blinded
experimental design, in which the provision of AI-generated recommendations
is randomized across cases with a human making final decisions. Under this
experimental design, we show how to compare the performance of three
alternative decision-making systems--human-alone, human-with-AI, and
AI-alone. We apply the proposed methodology to the data from our own
randomized controlled trial of a pretrial risk assessment instrument. We
find that AI recommendations do not improve the classification accuracy of
a judge's decision to impose cash bail. Our analysis also shows that
AI-alone decisions generally perform worse than human decisions with or
without AI assistance. Finally, AI recommendations tend to impose cash bail
on non-white arrestees more often than necessary when compared to white
arrestees.
The paper is available on arXiv:
https://arxiv.org/pdf/2403.12108.pdf
<2023-2024 Schedule>
GOV 3009 Website:
https://projects.iq.harvard.edu/applied.stats.workshop-gov3009
Calendar:
https://calendar.google.com/calendar/u/0?cid=Y18zdjkzcGF2OWZqa2tsZHJidTlzbm…
Best,
Jialu
--
Jialu Li
Department of Government
Harvard University
*https://jialul.github.io/ <https://jialul.github.io/>*