This was part of Reinforcement Learning from Offline Data and Human Feedback

Toward efficient exploration for language models

Dylan Foster, Microsoft Research