A generative model used in a scientific/engineering application should ideally be consistent with and leverage any low-dimensional structure arising from the underlying physical system. For instance, the generated samples must lie on the low-dimensional level sets of the conserved quantities so that the samples are physically realizable. The generative samples must be usable to learn this invariant manifold or the support of the target distribution. Finally, it is desirable to reduce the data and/or computational complexity of learning the generative model if the target distribution indeed has a low-dimensional support. In this talk, we will investigate these desirable properties of generative models and discuss sufficient conditions on score-based or flow-matching based generative models for when they are satisfied. To do this, we will treat generative models as random dynamical systems. This allows us to study propagation of uncertainties and probability flows along the sample trajectories, and provides clues on how to perform dimension reduction and control of generative models.