Permutation and Causal Inference

We propose a reproducing kernel Hilbert space approach for statistical inference regarding the slope in a function-on-function linear regression via penalised least squares, regularized by the thin-plate spline smoothness penalty. We derive a Bahadur expansion for the slope surface estimator and prove its weak convergence as a process in the space of all continuous functions. As a consequence of these results, we construct minimax optimal estimates, simultaneous confidence regions for the slope surface and simultaneous prediction bands Moreover, we derive new tests for the hypothesis that the maximum deviation between the “true” slope surface and a given surface is less or equal than a given threshold. In other words, we are not trying to test for exact equality (because in many applications this hypothesis is hard to justify), but rather for pre-specified deviations under the null hypothesis. To ensure practicability, non-standard bootstrap procedures are developed addressing particular features that arise in these testing problems. We also demonstrate that the new methods have good finite sample properties by means of a simulation study and illustrate their practicability by analyzing a data example.

Randomized experiments balance all covariates on average and provide the gold standard for estimating treatment effects. Chance imbalances nevertheless exist more or less in realized treatment allocations, complicating the interpretation of experimental results. To inform readers of the comparability of treatment groups at baseline, modern scientific publications often report covariate balance tables with not only covariate means by treatment group but also the associated p-values from significance tests of their differences. The practical need to avoid small p-values as indicators of poor balance motivates balance check and rerandomization based onthese p-values from covariate balance tests (ReP) as an attractive tool for improving covariate balance in randomized experiments. Despite the intuitiveness of such strategy and its possibly already widespread use in practice, the existing literature lacks results about its implications on subsequent inference, subjecting many effectively rerandomized experiments to possibly inefficient analyses. To fill this gap, we examine a variety of potentially useful schemes for ReP and quantify their impact on subsequent inference. Specifically, we focus on three estimators of the average treatment effect from the unadjusted, additive, and fully interacted linear regressionsof the outcome on treatment, respectively, and derive their asymptotic sampling properties under ReP. The main findings are threefold. First, ReP improves covariate balance between treatment groups, thereby strengthening the causal conclusions that can be drawn from experimental data. In addition to increasing comparability of treatment groups, the improved balance also reduces the asymptotic conditional biases of the estimators and ensures more coherent inferences between covariate-adjusted and unadjusted analyses. Second, the estimator from the fully interacted regression is asymptotically the most efficient under all ReP schemes examined, and permits convenient regression-assisted inference identical to that under complete randomization. Third, ReP improves the asymptotic efficiency of the estimators from the unadjusted and additive regressions. The corresponding standard regression analyses are accordingly still valid but in general overconservative. As a result, the combination of ReP for treatment allocationand fully interacted regression for analysis ensures both covariate balance and convenient and efficient inference. Importantly, our theory is design-based and holds regardless of how well the models involved in both the rerandomization and analysis stages represent the true data-generating processes.

This paper studies permutation tests for dependent data. Under a weak dependence structure, we prove the asymptotic validity of block-wise permutation tests using studentization and the self- normalized test statistics, where the block size is not a function of the sample size. Monte Carlo simulation exercises confirm that both the studentized and the self- normalized block-wise permutation tests have the correct test sizes with moderate degrees of dependence.

Competing approaches to inference in randomized experiments differ primarily in (1) which notion of “no treatment effect’’ being tested; and (2) whether or not stochasticity is assumed in the potential outcomes and covariates. Recommended hypothesis tests in a given paradigm may be invalid even asymptotically when applied in other frameworks, creating the risk of misinterpretation by practitioners when a given method is deployed. We develop a general framework for ensuring validity across competing modes of inference. We first describe a nested collection of bootstrap resampling schema providing valid inference of average treatment effects at differing levels of assumed stochasticity, ranging from superpopulation models, assuming random potential outcomes and covariates, to finite population inference, where only the assignment is viewed as random. To provide exact inference for stronger notions of no effect (such as Fisher’s sharp null), we then employ permutation tests based upon prepivoted test statistics, wherein a test statistic is first transformed by a particular bootstrap cumulative distribution function and its permutation distribution is then enumerated. This provides a single mode of inference which is exact for sharp nulls, asymptotically valid for average treatment effects at the specified level of stochasticity, with higher order improvements for inference in superpopulation models. 

We consider a vector of N independent binary variables, each with a different probability of success. The distribution of the vector conditional on its sum is known as the conditional Bernoulli distribution. Assuming that N goes to infinity and that the sum is proportional to N, exact sampling costs order N^2, while a simple Markov chain Monte Carlo algorithm using ‘swaps’ has constant cost per iteration. We provide conditions under which this Markov chain converges in order NlogN iterations. Our proof relies on couplings and an auxiliary Markov chain defined on a partition of the space into favorable and unfavorable pairs. This talk is based on joint work with Jeremy Heng and Pierre Jacob.

This talk will introduce two new tools for summarizing a probability distribution more effectively than independent sampling or standard Markov chain Monte Carlo thinning:

1. Given an initial n point summary (for example, from independent sampling or a Markov chain), kernel thinning finds a subset of only square-root n points with comparable worst-case integration error across a reproducing kernel Hilbert space.
2. If the initial summary suffers from biases due to off-target sampling, tempering, or burn-in, Stein thinning simultaneously compresses the summary and improves the accuracy by correcting for these biases.

These tools are especially well-suited for tasks that incur substantial downstream computation costs per summary point like organ and tissue modeling in which each simulation consumes 1000s of CPU hours.

Based on joint work with Raaz Dwivedi, Marina Riabiz, Wilson Ye Chen, Jon Cockayne, Pawel Swietach, Steven A. Niederer, Chris. J. Oates, Abhishek Shetty, and Carles Domingo-Enrich:

• Kernel Thinning (arXiv:2105.05842)
• Optimal Thinning of MCMC Output (arXiv:2005.03952)
• Generalized Kernel Thinning (arXiv:2110.01593)
• Distribution Compression in Near-linear Time (arXiv:2111.07941)
• Compress Then Test: Powerful Kernel Testing in Near-linear Time (arXiv:2301.05974)