Tencent Open Sources Hunyuan-A13B: A 13B Active Parameter MoE Model with Dual-Mode Reasoning and 256K Context

Tencent's Hunyuan team has unveiled Hunyuan-A13B, an open-source large language model leveraging a sparse Mixture-of-Experts (MoE) architecture that efficiently balances performance and computational cost by activating only 13 billion parameters out of 80 billion during inference. The model incorporates advanced features such as Grouped Query Attention (GQA), a 256K token context window, and a dual-mode reasoning framework that switches between fast and slow thinking modes, enhancing its capability for complex reasoning and long-context tasks. Built with a fine-grained MoE design, Hunyuan-A13