New AI Method From Meta and NYU Boosts LLM Alignment Using Semi-Online Reinforcement Learning

Meta and NYU have developed a novel semi-online reinforcement learning approach to enhance the alignment of large language models (LLMs) with human preferences, addressing the limitations of traditional offline and online methods. This technique enables LLMs to adapt more effectively during the fine-tuning process by leveraging human feedback, thereby improving their performance in instruction-based and mathematically precise tasks while balancing computational efficiency and adaptability. The new method builds upon existing alignment algorithms such as Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO), integrating semi-online strategies to optimize decision-making based on human preferences in real-time