Applications of Mean Field Games

We present an analysis of the Proof-of-Work (PoW) consensus algorithm, used on the Bitcoin blockchain, using a Mean Field Game framework. Using a master equation, we provide an equilibrium characterization of the total computational power devoted to mining the blockchain (hashrate). This new class of models allows us to adjust to many different situations. The PoW consensus algorithm has an intrinsic stability that makes it very suitable for creating a strong form of consensus. We show that a homogenous version of the model is able to explain well the Bitcoin data. In equilibrium, the revenue per unit of real hashrate follows a mean reverting process even with an unbounded reward process. We also show that, in the long run, R&D expenses, as well as the blockchain security and the bitcoin energy consumption, are all proportional to the total miners reward (i.e. the bitcoin price).

In this talk, we study the optimal control problem arising from the mean-field game formulation of the collective decision-making in honeybee swarms. A population of homogeneous players (the honeybees) has to reach consensus on one of two options. We consider three states: the first two represent the available options (or strategies), and the third one represents the uncommitted state. We formulate the continuous-time discrete-state mean-field game model. The contributions of this talk are the following: i) we propose an optimal control model where players have to control their transition rates to minimize a running cost and a terminal cost, in the presence of an adversarial disturbance; ii) we develop a formulation of the micro-macro model in the form of an initial-terminal value problem (ITVP) with switched dynamics; iii) we study the existence of stationary solutions and the mean-field Nash equilibrium for the resulting switched system; iv) we show that under certain assumptions on the parameters, the game does not admit Nash equilibrium solutions but only periodic solutions; and v) we analyze the resulting microscopic dynamics in a structured environment where a finite number of players interact through a network topology.

We propose a mean field game model to study the question of how centralization of reward and computational power occur in Bitcoin-like cryptocurrencies. Miners compete against each other for mining rewards by increasing their computational power. This leads to a novel mean field game of jump intensity control, which we solve explicitly for miners maximizing exponential utility, and handle numerically in the case of miners with power utilities. We show that the heterogeneity of their initial wealth distribution leads to greater imbalance of the reward distribution, and increased wealth heterogeneity over time, or a “rich get richer” effect. This concentration phenomenon is aggravated by a higher bitcoin mining reward, and reduced by competition. Additionally, an advantaged miner with cost advantages such as access to cheaper electricity, contributes a significant amount of computational power in equilibrium, unaffected by competition from less efficient miners. Hence, cost efficiency can also result in the type of centralization seen among miners of cryptocurrencies.

This talk presents recent advances on the design of Machine learning based approximation algorithms for Nash equilibrium in Mean Field games. In particular, a focus will be given on the one using Fictitious Play or Online Mirror Descent iterative schemes. A dedicated open source library, Open Spiel, together with applications to flocking, routing or crowd movements will be presented.

We consider the problem of reducing the carbon emissions of a set of firms over a finite horizon. A regulator dynamically allocates emission allowances to each firm. Firms face idiosyncratic as well as common economic shocks on emissions, and have linear quadratic abatement costs. Firms can trade allowances so to minimise total expected costs, from abatement and trading plus a quadratic terminal penalty. Using variational methods, we exhibit in closed-form the market equilibrium in function of regulator’s dynamic allocation. We then solve the Stackelberg game between the regulator and the firms. Again, we obtain a closed-form expression of the dynamic allocation policies that allow a desired expected emission reduction. Optimal policies are not unique but share common properties. Surprisingly, all optimal policies induce a constant abatement effort and a constant price of allowances. Dynamic allocations outperform static ones because of adjustment costs and uncertainty, in particular given the presence of common shocks. Our results are robust to some extensions, like risk aversion of firms or different penalty functions.

In a renewable energy certificate (REC) market a regulatory body imposes a lower bound on the amount of energy each regulated firm must generate via renewable means, providing them with a certificate for each MWh generated. Regulated firms seek to navigate the market to minimize the cost imposed on them, by modulating their REC generation and trading activities. As such, the REC market can be viewed through the lens of a large stochastic game with heterogeneous agents, where agents interact through the market price of the certificates. We study this stochastic game by solving the mean-field game (MFG) limit with sub-populations of heterogeneous agents. Moreover, we endogenize REC prices through market clearing. Using techniques from variational analysis, we characterize firms’ optimal controls as the solution of McKean-Vlasov (MV) FBSDEs and determine the equilibrium REC price. We establish the existence and uniqueness of a solution to this MV-FBSDE, and further prove that the MFG strategies have the Epsilon-Nash property for the finite player game. Finally, we develop a numerical scheme for solving the MV-FBSDEs and conclude by demonstrating how firms behave in equilibrium using simulated examples.

As the level of penetration of intermittent renewable energy sources increases in power systems, energy supply side reliability decreases. In this context, the flexibility afforded by electric loads associated with energy storage such as space heaters, air conditioners or water heaters becomes a crucial asset as these loads can be deferred or anticipated to mitigate energy supply variability. We consider the case of an aggregator coordinating a very large number of small residential electric space heating loads. The aggregator pursues two possible distinct objectives over a fixed time interval: (i) Reducing/increasing the loads mean energy requirements; (ii) Achieving a fixed mean load power demand rate. Both control problems are approached via a mean field prescriptive linear quadratic (LQ) game framework. (i) is solved using a nonstandard LQ game with cost coefficients involving a time integral of an error signal. (ii) is solved using a novel inverse mean field equilibrium approach. Based on joint work with A. Kizilkale, R. Salhab and Q. Lénet.

We develop the linear programming approach to mean-field games in a general setting. This relaxed control approach allows to prove existence results under weak assumptions, and lends itself well to numerical implementation. We consider mean-field game problems where the representative agent chooses both the optimal control and the optimal time to exit the game, where the instantaneous reward function and the coefficients of the state process may depend on the distribution of the other agents. Furthermore, we establish the equivalence between mean-field games equilibria obtained by the linear programming approach and the ones obtained via the controlled/stopped martingale approach, another relaxation method used in a few previous papers in the case when there is only control (joint work with M. Leutscher and P. Tankov)

From the beginning, mean field games have been used to model economic production for a market with a large number of producers. The master equation is an infinite dimensional PDE that in principle allows us to solve the convergence problem for N-player equilibria as N goes to infinity. However, up until now, there are no results in the literature on the master equation that satisfy the assumptions needed for economic production. In the paper I will discuss, co-authored with Ronnie Sircar, we prove the existence and uniqueness of solutions to a master equation corresponding to a mean field Cournot game, with absorbing boundary conditions.

I will describe a dynamic mean-field game on a dense heterogeneous directed network (graphon) which models the propagation of default losses in an interconnected financial system. In this game, each agent (a node of the network) chooses her outgoing (lending) connections strategically, in order to maximize the expected value of her future cash flow. A default of any agent shifts the values of her creditors downward. It is well known that without the strategic behavior such systems exhibit blow-ups — when a non-zero fraction of the population default at the same time. Remarkably, in equilibrium, the agents choose to adjust their strategies in order to minimize their exposure to the most vulnerable counter-parties, and the blow-ups never occur. On the mathematical side, the main challenge of this work is in constructing a mean-field equilibrium (which is also the “minimal” equilibrium) in this game, which is split into a dynamic and a static parts. The static part corresponds to constructing an equilibrium in a decentralized network-flow game, which is interesting in its own right. In particular, our solution to the static problem allows one to determine the distribution of a capital injection across the financial institutions so that the overall flow of capital to the real economy is maximized. Based on a joint work with M. Shkolnikov.

We investigate the formation of the price of an asset negotiated in an order book. In particular, we propose a game-theoretic model in many-person games and infinite games, considering costs stemming from limited liquidity. We derive closed-form formulas for the price in terms of the realized order flow. We also identify appropriate conditions that ensure the convergence of the price we find in the finite population game to that of its mean-field counterpart. We numerically assess our results using high-frequency data from ten stocks listed in the NASDAQ stock exchange.

The coronavirus disease 2019 (COVID-19) pandemic is changing and impacting lives on a global scale. In this paper, we introduce a mean-field game model in controlling the propagation of epidemics on a spatial domain. The control variable, the spatial velocity, is first introduced for the classical disease models, such as the Suspected-infected-recovered model. For this proposed model, we provide fast numerical algorithms based on proximal primal-dual methods. Numerical experiments demonstrate that the proposed model illustrates how to separate infected patients in a spatial domain effectively.  This is based on joint work with Wonjun Lee, Siting Liu, Hamidou Tembine, Stanley Osher.

In this talk, we consider the control of the COVID-19 pandemic from two point of views: individual and governmental. In the first part, through a standard SIR compartmental model, we assume that the control is induced by the aggregation of individuals’ decisions to limit their social interactions: when the epidemic is ongoing, an individual can diminish his/her contact rate in order to avoid getting infected, but this effort comes at a social cost. If each individual lowers his/her contact rate, the epidemic vanishes faster, but the effort cost may be high. A Mean Field Nash equilibrium at the population level is formed, resulting in a lower effective transmission rate of the virus. We prove theoretically that equilibrium exists and compute it numerically. However, this equilibrium selects a sub-optimal solution in comparison to the societal optimum (a centralised decision respected fully by all individuals), meaning that the cost of anarchy is strictly positive. This lead us to consider, in the second part, the governmental point of view. In this model, the population is considered as a single agent, but the government can put into place incentive policies to encourage the lockdown. In addition, the government may also implement a testing policy in order to know more precisely the spread of the epidemic within the country, and to isolate infected individuals. Numerical results confirm the relevance of a tax and testing policy to improve the control of an epidemic. Joint works with Romuald Elie, Thibaut Mastrolia, Dylan Possamaï, Gabriel Turinici and Xavier Warin.

In this talk, we consider a Stackelberg mean field game model between a principal and a mean field of agents evolving on a finite state space, motivated by models of epidemic control in large populations. The agents play a non-cooperative game in which they can control their transition rates between states to minimize an individual cost. The principal can influence the resulting Nash equilibrium through incentives to optimize its own objective. Later, we propose an application to an epidemic model of SIR type in which the agents control their interaction rate and the principal is a regulator acting with non pharmaceutical interventions. To compute the solutions, we use an innovative numerical approach based on Monte Carlo simulations and machine learning tools for stochastic optimization. Finally, we briefly discuss another game formulation for a continuum of non-identical players evolving on a finite state space where their interactions are represented by a graphon.