Abstract: Mean-field control and mean-field game models provide a powerful framework for studying decision-making in systems with a large number of interacting agents. This talk introduces multi-scale, model-free reinforcement learning algorithms that separate the learning of individual policies, value functions, and population-level distributions across distinct time scales. This perspective captures the coupled evolution of agents and the mean field. The talk will focus on empirical results for Q-learning and actor–critic methods, illustrating how multi-time-scale structure influences performance in mean-field control, game, and mixed settings, including discrete and continuous state spaces as well as finite- and infinite-horizon problems.