Causal Inference in Observational Studies

Presenter: Tom Leavitt

Causal Inference in Observational Studies: Assignment Mechanism Models and the Role of Covariate Balance

Observational studies posit that units’ treatment assignment probabilities are governed by an unknown function of baseline covariates such that, if units have identical baseline covariates, then they must have identical treatment assignment probabilities. But even if all potentially confounding covariates are observed, exact balance on all of them is both mathematically and practically infeasible; hence, researchers often conduct tests of covariate balance to assess whether a given design has “enough” balance such that the study can be analyzed as if it is a block, uniform randomized experiment. In this presentation, I explain that the null hypothesis of such tests is not covariate balance, per se, but the proposition that treatment assignment probabilities are uniformly distributed within covariate blocks. I then formally demonstrate that tests of this null hypothesis are biased against a subset of alternative hypotheses that probabilities are nonuniformly distributed within blocks, and subsequently develop a power analysis that enables researchers to discern the specific alternative hypotheses for which the aforementioned null hypothesis is biased and unbiased, respectively. The argument and method advanced in this presentation have the potential to shed light on debates about the role that models of the assignment mechanism and subsequent assessments of covariate balance play in drawing causal inferences from observational data.