Introduction to Decision Making and Uncertainty

Week 1: June 28-July 2

Foundations on stochastic analysis, stochastic optimization, BSDE and applications

Organizer: Thaleia Zariphopoulou (McCombs Business School and Mathematics, UT Austin) 

This tutorial will provide an introduction to various methodological areas that are central in decision making under uncertainty. Specifically, they will cover foundational material in stochastic optimization, stochastic analysis and its connection with pde. They will also provide an introduction to backward stochastic differential equations (BSDE) and BSDE systems. In addition, they will offer an introduction to functional Itô calculus and its applications, as well as an introduction to robo-advising as a human-machine stochastic interactive system.

Monday, June 28

Title: Foundations of stochastic optimization, Dynamic Programming and HJB equations

Speaker: Thaleia Zariphopoulou (UT-Austin) 

Tuesday, June 29

Title:  Probabilistic methods for elliptic and parabolic PDEs: from linear equations to free-boundary problems

Speaker: Sergey Natdochiy (IIT) 

Wednesday, June 30

Title: Foundations of Backward Stochastic Differential Equations and their applications

Speakers:   Gordan Zitkovic (UT-Austin) and Joseph Jackson (UT-Austin)   

Thursday, July 1

Title: Introduction to Functional Itô Calculus and applications

Speaker: Rama Cont  (Oxford University) 

Friday, July 2

Speakers:  Agostino Capponi (Columbia University) and Sveinn Olafsson (Columbia University)

Title: Introduction to robo-advising as a human-machine interaction system: modeling and learning

Week 2:  July 5-9

Monday, July 5  Holiday

July 6-7  

Optimal transport and machine learning 

Organizer: Marcel Nutz (Mathematics and Statistics, Columbia University)

The four parts of this tutorial will cover the mathematical foundations of optimal transport, modern computational approaches and applications in machine learning, sampling properties of optimal transport, and applications of transport maps in nonparametric statistics.

Speakers:                                

Aude Genevay (MIT)      

Jonathan Niles-Weed (New York University)

Marcel Nutz (Columbia University)

Bodhisattva Sen (Columbia University)

July 8-9 

Time-inconsistent  and relaxed stochastic optimization, and  applications

Organizer: Xunyu Zhou (Columbia University)

This tutorial will provide an introduction to the theory of relaxed controls and their applications to reinforcement learning, including decision making under exploration via randomization in continuous time. The tutorial will also provide an introduction to time-inconsistent stochastic optimization and its applications to behavioral finance under decision making criteria that involve rank-dependent risk preferences, hyperbolic discounting, distorted probabilities and non-linear expectations. 

Thursday, July 8

Speakers:  Xunyu Zhou (Columbia University) and Wenpin Tang (Columbia University) 

Title: Exploration via Randomization: Reinforcement Learning and Beyond

Friday, July 9

Speakers:  Xuedong He (CUHK) and Moris Strub (SUSTECH)  

Title: Foundations of time-inconsistent control, and applications to decision making under non-standard optimality criteria

Week 3: July 12-16

Markov decision processes with dynamic risk measures: optimal control and learning 

Organizers: Tomasz Bielecki (IIT) and Andrzej Ruszczyński (Department of Management Science and Information Systems, Rutgers)

Lecture 1: Foundations of Dynamic Risk Measurement (Andrzej Ruszczynski)

We shall discuss the background of the theory of measures of risk, their main properties, and examples. Special attention will be paid to the dual representation, law-invariance, and the Kusuoka representation. Then we shall discuss issues associated with measuring risk of sequences, in particular: time consistency and the local property. We shall extend the dual representation to dynamic risk measures. Finally, we shall discuss risk measurement in continuous time.

Lecture 2: Risk Sensitive Markov Decision Processes (Tomasz Bielecki)

We shall relate the risk sensitive criterion to the entropic risk measure. Some relevant properties of the risk sensitive criterion will be presented. A discussion of Markov decisions processes will follow, mostly in discrete time. A study of a finite time horizon risk sensitive MDP subject to model uncertainty will be presented as well.

Lecture 3: Markov Control Problems with Markov Risk Measures (Andrzej Ruszczynski)

We shall adapt the theory of dynamic risk measurement to Markov systems and we shall introduce the concept of a Markov risk measure. We shall study the properties of such measures and develop optimality conditions and methods for three classes of control problems: finite horizon, infinite horizon with discount, and infinite horizon for transient systems. Finally, we shall mention two continuous-time models: a controlled jump process and a controlled diffusion process.

Lecture 4:  Risk-Averse Reinforcement Learning (Andrzej Ruszczynski)

We shall discuss large-scale risk-averse Markov control problems with the use of value function approximations by a linear function of state features. We construct a projected risk-averse dynamic programming equation and study its properties.  Then we shall present risk-averse counterparts of the basic and multi-step methods of temporal differences and discuss their convergence with probability one. Finally, we shall mention a risk-averse SARSA approach with the use of Q-factors.

Day 5: Workshop 

Alexander Shapiro (Industrial and Systems Engineering at Georgia Tech)

Darinka Dentcheva (Mathematical Sciences, Stevens Institute of Technology) 

Mert Gurbuzbalaban (Department of Management Science and Information Systems, Rutgers Business School)

Igor Cialenco  (Applied Mathematics, IIT) 

July 12-16: Machine learning and Mean Field Games 

Xin Guo  (IEOR, UC Berkeley)

Lecture 1

Title: Generative Adversarial Networks: An Optimal Transport and Game Perspective

Abstract: Generative Adversarial Networks  (GANs) have enjoyed tremendous success in image generation and processing, and have recently attracted growing interests in financial modelings. In this tutorial we will introduce  GANs from the perspective of mean field games (MFGs) and optimal transport (OT).  We will first discuss the well-posedness of GANs as a minmax game, and then the variational structure of GANs, as well as GANs’ connection with mean-field games . We will next demonstrate how this game perspective enables a GANs-based algorithm (MFGANs) to solve efficiently high-dimensional mean-field games: the two neural networks trained in an adversarial way.

This new perspective will naturally lead to an analytical connection between GANs and Optimal Transport (OT) problems, for which we will provide sufficient conditions for the minimax games of GANs to be reformulated in the framework of OT.

Based on joint works with Cindy (Haoyang) Cao of Alan Turing Institute and Mathieu Lauriere of Princeton University.   

Lecture II

Title: Convergence of GANs training: An Stochastic Approximation and Control Approach 

Abstract: Despite the popularity of Generative Adversarial Networks (GANs), there are well recognised and documented issues for GANs training. In this second part of the tutorial, we will first introduce a stochastic differential equation approximation approach for GANs training.  We will next  demonstrate the connection of this SDE approach with the classical Newton’s method, and then show how this approach will enable studies of the convergence of GANs  training,  as well as  analysis of the hyperparameters training for GANs in a stochastic  control framework.  

Based on joint works with Cindy (Haoyang) Cao of Alan Turing Institute and Othmane Mounjid of UC Berkeley. 

Dacheng Xiu (Booth, University of Chicago)

“Predicting returns with text data” and “(Re-)Imag(in)ing Price Trends.” 

The common theme of these two talks is on machine learning applications to investment with alternative data. Compared to the other speakers, my talks will be rather applied. 

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3389884

We introduce a new text-mining methodology that extracts information from news articles to predict asset returns. Unlike more common sentiment scores used for stock return prediction (e.g., those sold by commercial vendors or built with dictionary-based methods), our supervised learning framework constructs a score that is specifically adapted to the problem of return prediction. Our method proceeds in three steps: 1) isolating a list of terms via predictive screening, 2) assigning prediction weights to these words via topic modeling, and 3) aggregating terms into an article-level predictive score via penalized likelihood. We derive theoretical guarantees on the accuracy of estimates from our model with minimal assumptions. In our empirical analysis, we study one of the most actively monitored streams of news articles in the financial system–the Dow Jones Newswires–and show that our supervised text model excels at extracting return-predictive signals in this context. Information in newswires is assimilated into prices with an inefficient delay that is broadly consistent with limits-to-arbitrage (i.e., more severe for smaller and more volatile firms) yet can be exploited in a real-time trading strategy with reasonable turnover and net of transaction costs.

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3756587

We reconsider the idea of trend-based predictability using methods that flexibly learn price patterns that are most predictive of future returns, rather than testing hypothesized or pre-specified patterns (e.g., momentum and reversal). Our raw predictor data are images—stock-level price charts—from which we elicit the price patterns that best predict returns using machine learning image analysis methods. The predictive patterns we identify are largely distinct from trend signals commonly analyzed in the literature, give more accurate return predictions, translate into more profitable investment strategies, and are robust to a battery of specification variations. They also appear context-independent: Predictive patterns estimated at short time scales (e.g., daily data) give similarly strong predictions when applied at longer time scales (e.g., monthly), and patterns learned from US stocks predict equally well in international markets.

Huyen Pham  (Université de Paris Diderot)

Lecture 1

Title: Control of McKean-Vlasov systems and applications

This lecture is concerned with the optimal control of McKean-Vlasov equations, which has been knowing a surge of interest since the emergence of the mean-field game theory. Such control problem corresponds to the asymptotic formulation of a N-player cooperative game under mean-field interaction, and can also be viewed as an influencer strategy problem over an interacting large population. It finds various applications in economy, finance, or social sciences for modelling motion of socially interacting individuals and herd behavior. It is also relevant for dealing with intermittence questions arising typically in risk management.

In the first part, I will focus on the discrete-time case, which extends the theory of Markov decision processes (MDP) to the mean-field interaction context. We give an application with explicit results to a problem of targeted advertising via social networks.

The second part is devoted to the continuous-time framework. We shall first consider the important class of linear-quadratic McKean-Vlasov (LQMKV) control problem, which provides a major source for examples and applications. We show a direct and elementary method for solving explicitly LQMKV based on a mean version of the well-known martingale optimality principle in optimal control, and the completion of squares technique. Next, we present the dynamic programming approach (in other words, the time consistency approach) for the control of general McKean-Vlasov dynamics. In particular, we introduce the recent mathematical tools that have been developed in this context : differentiability in the Wasserstein space of probability measures, Itô formula along a flow of probability measures and Master Bellman equation. 

Lecture 2

Title: Deep learning algorithms for mean-field control problems 

Machine learning methods for solving nonlinear partial differential equations (PDEs) and control problems  are hot topical issues, and different algorithms proposed in the literature  show efficient numerical approximation  in high dimension. This lecture will present recent deep learning schemes  for solving mean-field control problem, and the corresponding PDEs in Wasserstein space of probability measures. Some numerical tests for the examples of a mean-field systemic risk,  mean-variance problem

Christa Cucherio (Statistics and Operations Research, University of Vienna)

Title: From neural SDEs and signature methods to affine processes and back

Abstract: Modern universal classes of dynamic processes, based on neural networks or signature methods, have recently entered the field of stochastic modeling, in particular in Mathematical Finance. This has opened the door to more data-driven and thus more robust model selection mechanisms, while first principles like no arbitrage still apply. The underlying model classes are often so-called neural stochastic differential equations (SDEs) or signature SDEs, i.e. SDEs whose characteristics are either neural networks or linear functions of the process’ signature. We present methods how to learn these characteristics from available option price and time series data.

From a more theoretical point of view, we show how these new models can be embedded in the framework of affine and polynomial processes, which have been — due to their tractability — the dominating process class prior to the new era of highly over-parametrized dynamic models. Indeed, we prove that generic classes of diffusion models can be viewed as infinite dimensional affine processes, which in this setup coincide with polynomial processes. A key ingredient to establish this result is again the signature process. This then allows to get  power series expansions for expected values of analytic functions of the process’ marginals, which also apply to neural or signature SDEs.  In particular, expected signature can be computed via polynomial technology.

July 19-23:  Models for climate change with ambiguity and misspecification concerns 

Organizer: Lars Hansen (Economics, University of Chicago)

Two part lecture: “Decision Theory Tools for Uncertainty, Including Ambiguity and Misspecification Concerns,” –  Massimo Marinacci (Professor in the Department of Decision Sciences of Bocconi University) [Note: Shared with the “Models of Climate Change with Ambiguity and Misspecification” tutorial]

Two part lecture: “Dynamic Decision Theory Under Uncertainty: Tools and Applications,” – Lars Peter Hansen (Economics, University of Chicago)

Two part lecture: “Economics of Climate Change in the Face of Uncertainty,” – William A. Brock (Economics, University of Wisconsin-Madison) 

Two part lecture: “Solving and Assessing Economic Models of Climate Change and Accounting for Uncertainty” – Michael Barnett (Department of Finance, Arizona State University)

July 19-23:  Games with ambiguity 

Organizer: Peter Klibanoff (Managerial Economics & Decision Sciences, Kellogg School of Management, Northwestern University)

The tutorial will provide an introduction to some recent approaches to game theory (the theory of strategic interactions) when players may be averse to ambiguity (subjective uncertainty about probabilities). It includes lectures on decision-theoretic models of ambiguity-averse preferences and theories of how to update such preferences in a dynamically consistent manner after observing new information, in addition to lectures on game theory and mechanism design with these ambiguity concerns.

Two part lecture: “Decision Theory Tools for Uncertainty, Including Ambiguity and Misspecification Concerns,” –  Massimo Marinacci (Department of Decision Sciences, Bocconi University)  [Note: Shared with the “Models of Climate Change with Ambiguity and Misspecification” tutorial]

Two part lecture: “Dynamically Consistent Updating, Including under Ambiguity,” – Eran Hanany (Industrial Engineering, Tel Aviv University)

Two part lecture: “Games with Ambiguity,” – Peter Klibanoff (Managerial Economics & Decision Sciences, Kellogg School of Management, Northwestern University, 

One part lecture: “Mechanism Design with Ambiguity,” – Sujoy Mukerji (Economics and Finance, Queen Mary University of London)