Abstract: Reinforcement learning (RL) is an essential method for training large language models (LLMs), enabling better alignment with human preferences and enhanced reasoning capabilities. Nevertheless, RL post-training remains computationally demanding due to repeated rollouts, high-variance credit assignment, and the complexities of distributed systems that can introduce policy lag. A promising direction is to view KL-regularized policy optimization as a KL-prox (mirror-descent–style) step and solve it with a simple least-squares regression loss. This regression-based perspective addresses key RL challenges and enables more efficient post-training procedures. In this talk, I will introduce a unified perspective encompassing three complementary approaches. A⋆-PO minimizes reliance on online sampling by utilizing offline value computation and optimal-advantage regression. OAPL supports scalable, fully off-policy regression, even when using stale rollouts and lagged policies in distributed settings. RDA2C offers a regularized dual-averaging approach that employs cumulative gradient information to stabilize updates and reduce variance, rather than relying on local, per-round mirror-descent steps. Although RDA2C has been primarily assessed on standard RL benchmarks rather than comprehensive LLM post-training, its focus on variance reduction and data reuse aligns closely with the stability challenges encountered in large-scale LLM alignment. Collectively, these methods provide an efficient toolkit for RL-based LLM post-training and present opportunities for further research and scalable deployment.