Sakana AI Introduces Reinforcement-Learned Teachers (RLTs): Efficiently Distilling Reasoning in LLMs Using Small-Scale Reinforcement Learning

Sakana AI has developed Reinforcement-Learned Teachers (RLTs), a novel framework that enhances reasoning capabilities in language models by training smaller models as optimized instructors through a reinforcement learning approach focused on pedagogical explanations. Unlike traditional RL methods that rely on sparse reward signals and high computational costs, RLTs utilize dense, student-aligned rewards by prompting models with both problems and solutions, enabling the generation of detailed reasoning traces that improve distillation quality, cost-efficiency, and transferability across domains. This innovative approach redefines the teacher-student paradigm by emphasizing teaching rather than problem-solving, allowing smaller