Advances in Distribution Compression
Lester Mackey, Microsoft New England
This talk will introduce two new tools for summarizing a probability distribution more effectively than independent sampling or standard Markov chain Monte Carlo thinning:
- Given an initial n point summary (for example, from independent sampling or a Markov chain), kernel thinning finds a subset of only square-root n points with comparable worst-case integration error across a reproducing kernel Hilbert space.
- If the initial summary suffers from biases due to off-target sampling, tempering, or burn-in, Stein thinning simultaneously compresses the summary and improves the accuracy by correcting for these biases.
These tools are especially well-suited for tasks that incur substantial downstream computation costs per summary point like organ and tissue modeling in which each simulation consumes 1000s of CPU hours.
Based on joint work with Raaz Dwivedi, Marina Riabiz, Wilson Ye Chen, Jon Cockayne, Pawel Swietach, Steven A. Niederer, Chris. J. Oates, Abhishek Shetty, and Carles Domingo-Enrich:
- Kernel Thinning (arXiv:2105.05842)
- Optimal Thinning of MCMC Output (arXiv:2005.03952)
- Generalized Kernel Thinning (arXiv:2110.01593)
- Distribution Compression in Near-linear Time (arXiv:2111.07941)
- Compress Then Test: Powerful Kernel Testing in Near-linear Time (arXiv:2301.05974)