This was part of New Directions in Algebraic Statistics 

Advances in variational inference for singular models

Depdeep Pati, University of Wisconsin-Madison

Tuesday, July 22, 2025



Slides
Abstract:

The marginal likelihood or evidence in Bayesian statistics contains an intrinsic penalty for larger model sizes and is a fundamental quantity in Bayesian model comparison. Unlike regular models where the Bayesian information criterion (BIC) encapsulates a first-order expansion of the logarithm of the marginal likelihood, parameter counting gets trickier in singular models where a quantity called the real log canonical threshold (RLCT) summarizes the effective model dimensionality.  For complex singular models where the marginal likelihood is intractable, variational inference is often utilized to approximate an intractable marginal likelihood.  We show that mean-field variational inference correctly recovers the RLCT for any singular model in its canonical or normal form. We additionally exhibit sharpness of our bound by analyzing the dynamics of a general purpose coordinate ascent algorithm (CAVI) popularly employed in variational inference.  If a singular model is not in the normal form, we demonstrate that one can use a more flexible variational family using normalizing flows to recover the RLCT.