Causal Inference in the Age of Big Data

The rise of massive data sets that provide fine-grained information about human beings and their behavior offers unprecedented opportunities for evaluating the effectiveness of social, behavioral, and medical treatments. With the availability of fine-grained data, researchers and policymakers are increasingly unsatisfied with estimates of average treatment effects based on experimental samples that are unrepresentative of populations of interest. Instead, they seek to target treatments to particular populations and subgroups. Because of these inferential challenges, Machine Learning (ML) is now being used for evaluating and predicting the effectiveness of interventions in a wide range of domains from technology firms to clinical medicine and election campaigns. However, there are a number of issues that arise with the use of ML for causal inference. For example, although ML and related statistical models are good for prediction, they are not designed to estimate causal effects. Instead, they focus on predicting observed outcomes. In this talk, a number of meta-algorithms are presented that can take advantage of any supervised learning method to estimate the Conditional Average Treatment Effect function. Also, discussed are new theoretical results on confidence intervals and overlap in high-dimensional covariates and a new algorithm for optimal linear aggregation functions for tree-based estimators.

About the speaker: Jasjeet Sekhon is the Robson Professor of Political Science and Statistics at the University of California, Berkeley. His current research focuses on creating new machine learning methods for estimating causal effects in observational and experimental studies and evaluating social science, digital, public health, and medical interventions. He is also the Head of Causal Inference at Bridgewater Associates.

Reception to follow.