Scaling Offline RL via Efficient and Expressive Shortcut Models

The paper introduces SORL, a scalable offline reinforcement learning algorithm that utilizes novel shortcut generative models to enable efficient training and inference, capturing complex data distributions with a simple, one-stage process. SORL enhances policy performance across various offline RL tasks by employing sequential and parallel inference methods, verified by a learned Q-function, demonstrating positive scaling with increased test-time compute.