This was part of
Kernel Methods in Uncertainty Quantification and Experimental Design
On kernelized particle systems for sampling and quantization
Youssef Marzouk, MIT Center for Computational Science and Engineering
Wednesday, April 2, 2025
Abstract: Characterizing and summarizing complex probability distributions is a central task in applied mathematics, statistics, and machine learning with myriad applications. I will discuss kernel-based interacting particle systems (IPS) that solve two different instantiations of this problem. First is the problem of sampling from a target distribution whose unnormalized density, with respect to some reference distribution, is available. We introduce a mean-field ODE and a corresponding IPS that approximate a Fisher–Rao gradient flow from the reference to the target, and that can be integrated without access to gradients of the target density. This sampler is an instance of broader class of kernelized samplers; we will discuss key choices in the design of these schemes.
Second is the problem of weighted quantization, i.e., summarizing a complex distribution with a small set of weighted Dirac measures. We study this problem from the perspective of minimizing maximum mean discrepancy via gradient flow in the Wasserstein--Fisher--Rao (WFR) geometry. This gradient flow yields an ODE system from which we further derive a fixed-point algorithm called mean shift interacting particles (MSIP). We show that MSIP extends the (non-interacting) mean shift algorithm, widely used for identifying modes in kernel density estimates.
This is joint work with Aimee Maurais, Ayoub Belhadji, and Daniel Sharp.