This was part of Machine Learning for Climate and Weather Applications

Combining Stochastic Parameterized Reduced Order Models with Machine Learning for Data Assimilation and Uncertainty Quantification with Partial Observations

Nan Chen, University of Wisconsin, Madison

Monday, October 31, 2022



Abstract: Effective and efficient data assimilation is challenging for complex turbulent systems with partial observations. In this work, we propose a new hybrid nonlinear data assimilation framework for such systems. This hybrid framework contains four steps: (1). With the observed state variable data, we build a stochastic surrogate model using stochastic parameterization Kalman filters (SPEKF); (2). We train a set of long short-term memory (LSTM) neural networks (NN) between observed and unobserved variables; (3). We apply the ensemble transform Kalman filter (ETKF) to obtain the posterior mean of observed variables and use the filtered data as inputs in LSTM-NN from step (2) to recover the unobserved variables; and (4). We quantify the uncertainty for the recovered unobserved state variables using the efficient conditional Gaussian mixture filter. One application we considered is the recently proposed precipitation quasi-geostrophic (PQG) equations. Compared with classic quasi-geostrophic equations, PQG introduces an additional moisture variable and Heaviside nonlinearities to account for phase changes of water and thus is more complex in physics and expensive in computation. The numerical results show that compared with bare ETKF or localized (ETKF) (LETKF) scenarios, the new hybrid framework is more efficient and accurate in reproducing the dynamics and statistical features for both observed and unobserved state variables.