**Bayesian Statistics and Statistical Learning**

## Minimum distance estimators and inverse bounds for latent probability measures

**
Xuanlong Nguyen, University of Michigan
**

**Tuesday, December 12, 2023**

**Abstract**:

We study the learning of finite mixture modeling in a singular setting,

where the number of model parameters is unknown. Simply overfitting

without suitable regularization of the model can result in highly inefficient

parameter estimation behavior.

To achieve the optimal rate of parameter estimation, we propose a

general framework for estimating the mixing measure arising in

finite mixture models, which we term minimum $Phi$-distance estimators.

We establish a general theory for the general minimum $Phi$-distance

estimator, which involves obtaining sharp probability bounds on the

estimation error for the mixing measure in terms of the suprema of

the associated empirical processes for a suitably chosen function class

$Phi$. The theory makes clear the dual roles played by the function

class $Phi$, rich enough to induce necessary strong identifiability

conditions and small enough to produce optimal rates for parameter

estimation.

Our framework not only includes many existing estimation methods as special

cases but also results in new ones. For instance, it includes the minimum

Kolmogorov-Smirnov distance estimator as a special case, but also extends

to the multivariate setting. It includes the method of moments as well,

while extending existing results for Gaussian mixtures to a larger family

of probability kernels. Moreover, the minimum $Phi$-distance estimation

framework leads to new methods applicable to complex (e.g., non-Euclidean)

observation domains. In particular, we study a novel minimum distance

estimator based on the maximum mean discrepancy (MMD), a particular

$Phi$-distance that arises in a different context (of learning

with reproducing kernel Hilbert spaces).

This work is joint with Yun Wei and Sayan Mukherjee.