New Directions in Algebraic Statistics 

Preliminary abstract: I will highlight the role of algebraic geometry in enhancing our understanding of machine learning models, offering a fresh perspective on the problem of neural network verification. I will discuss an approach for ensuring the behavior of a ReLU network at test time by establishing systematic algebraic relations that are satisfied by the outputs produced at various data points. I will emphasize the combinatorial and geometric properties of these relations and explain how they constrain the possible output values at test points of interest. This is joint work with Guido Montúfar.

An important question that arises when modeling is if the unknown parameters of a model can be determined from real (and sometimes noisy) data, the so-called parameter estimation problem.  A key first step is to ask which parameters can be determined given perfect data, i.e. noise-free and of any time duration required.  This is called the structural identifiability problem.  If all of the parameters can be determined from data, we say the model is identifiable.  However, if there is some subset of parameters that can take on an infinite number of values yet yield the same data, we say the model is unidentifiable. If a model is unidentifiable assuming perfect data, then it is almost certainly unidentifiable with real, noisy data, thus knowing this information a priori helps with experimental design.  We examine this question for an important class of models called linear compartmental models used in many areas, such as pharmacokinetics, physiology, cell biology, toxicology, and ecology.  We also examine a somewhat related question called indistinguishability, which examines if two distinct models can yield the same dynamics.  In this case, two models with completely different structures can be indistinguishable from an input-output perspective.  On top of this, there are questions of observability, controllability, and model selection/rejection.  For all of these questions, we will consider the underlying graphs corresponding to our models and use tools from graph theory and computational algebra to describe and analyze our models.

The marginal likelihood or evidence in Bayesian statistics contains an intrinsic penalty for larger model sizes and is a fundamental quantity in Bayesian model comparison. Unlike regular models where the Bayesian information criterion (BIC) encapsulates a first-order expansion of the logarithm of the marginal likelihood, parameter counting gets trickier in singular models where a quantity called the real log canonical threshold (RLCT) summarizes the effective model dimensionality.  For complex singular models where the marginal likelihood is intractable, variational inference is often utilized to approximate an intractable marginal likelihood.  We show that mean-field variational inference correctly recovers the RLCT for any singular model in its canonical or normal form. We additionally exhibit sharpness of our bound by analyzing the dynamics of a general purpose coordinate ascent algorithm (CAVI) popularly employed in variational inference.  If a singular model is not in the normal form, we demonstrate that one can use a more flexible variational family using normalizing flows to recover the RLCT.

The motivation for this talk is to detect when an irreducible projective variety V is not toric. We do this by introducing and analyzing a symmetry Lie group and a Lie algebra associated with the ideal I(V). If the dimension of V is strictly less than the dimension of the above-mentioned objects, then there is no linear transformation that turns I(V) into a toric ideal. We use it to provide examples of non-toric statistical models in algebraic statistics.

The expectation maximization (EM) algorithm is a popular method of density estimation for Gaussian mixture models. Fundamental to understanding the performance of this algorithm is to understand the set of points closest to a given Gaussian, where “closest" is defined in terms of the maximum likelihood function. We call this set of points a Gaussian Voronoi cell. In this talk, I will outline new results regarding the geometry and combinatorics of Gaussian Voronoi cells. This is joint work with Joe Kileel.

Phylogenetic networks provide a framework for representing complex evolutionary histories involving processes such as hybridization and horizontal gene transfer. An important question in their study is whether the structure of a network (or which features of it) can, in principle, be recovered from genetic data. This talk discusses the identifiability of phylogenetic networks under the multispecies coalescent model. First, I review foundational results for a 'simple' class of networks, then present recent work that establishes identifiability for a much broader network class.

"We study the characterization, expressivity and learning of polynomial neural networks with monomial activation functions. In special cases, we describe neuromanifolds as semialgebraic sets and neurovarieties as algebraic varieties. We study the expressivity of polynomial neural networks by exploring their dimension. Finally, we define and investigate the learning degree, which is a characteristic of the optimization landscape of a polynomial neural network. 
This talk is based on joint work with Jiayi Li and Maximilian Wiesmann. It was done as part of the apprenticeship program ""Varieties from Statistics"" during the IMSI long program ""Algebraic Statistics and Our Changing World""."

This talk will describe recent results on the algebraic structure of phylogenetic network models under the displayed tree model, and connections to graphical models.  In particular, we show how the connection to graphical models yields new (non)identifiability results for the phylogenetic network models.