Work

Counterfactual inference in sequential experiments

Year: 2025

Type: article

Abstract: We consider after-study statistical inference for sequentially designed experiments wherein multiple units are assigned treatments for multiple time points using treatment policies that adapt over time. Our goal is to provide inference guarantees for the counterfactual mean at the smallest possible scale—mean outcome under different treatments for each unit and each time—with minimal assumptions on the adaptive treatment policy. Without any structural assumptions on the counterfactual means, this challenging task is infeasible due to more unknowns than observed data points. To make progress, we introduce a latent factor model over the counterfactual means that serves as a nonparametric generalization of the nonlinear mixed effects model and the bilinear latent factor model considered in prior works. For estimation, we use a nonparametric method, namely a variant of nearest neighbors, and establish a nonasymptotic high probability error bound for the counterfactual mean for each unit and each time. Under regularity conditions, this bound leads to asymptotically valid confidence intervals for the counterfactual mean as the number of units and time points grows to ∞ together at suitable rates. We illustrate our theory via several simulations and a case study involving data from a mobile health clinical trial HeartSteps.

Source: The Annals of Statistics

DOI: 10.1214/25-aos2519

Authors: Raaz Dwivedi, Katherine Tian, Sabina Tomkins, Predrag Klasnja, Susan Murphy, Devavrat Shah

Institutions: Cornell Tech, Harvard University, University of Michigan, Massachusetts Institute of Technology

Language: English

Topic: Advanced Causal Inference Techniques

Subfield: Statistics and Probability

Field: Mathematics

Domain: Physical Sciences

Open Access status: green