Abstract: Systematic discriminatory biases present in our society influence the way data is collected and stored, the way variables are defined, and the way scientific findings are put into practice as policy. Automated decision procedures and learning algorithms applied to such data may serve to perpetuate existing injustice or unfairness in our society.  We consider how to solve prediction and policy learning problems in a way which breaks the cycle of injustice'' by correcting for the unfair dependence of outcomes, decisions, or both, on sensitive features (e.g., variables that correspond to gender, race, disability, or other protected attributes).  We use methods from causal inference and constrained optimization to learn outcome predictors and optimal policies in a way that addresses multiple potential biases which afflict data analysis in sensitive contexts.  Our proposal comes equipped with the guarantee that solving prediction or decision problems on new instances will result in a joint distribution where the given fairness constraint is satisfied.  We illustrate our approach with both synthetic data and real criminal justice data.