Abstract: In the same way that computer vision models find structures and patterns in images, how might reinforcement learning models find structures and patterns in solutions to control problems? This talk will focus on learning temporal representations, which map high-dimensional observations to compact representations where distances reflect shortest paths. Once learned, these temporal representations encode the value function for certain tasks – learning temporal representations is itself an RL algorithm. In both robotics and reasoning problems, such representations capture temporal patterns. Temporal representations also facilitate a form of (temporal) generalization: navigating between pairs of states that are more distant than those seen during training. I will show evidence that agents trained via temporal representations exhibit surprising exploration strategies, in both single-agent and multi-agent settings.