Baidu Open Sources ERNIE 4.5: LLM Series Scaling from 0.3B to 424B Parameters

Baidu has open-sourced its ERNIE 4.5 series, a comprehensive family of foundation models designed for advanced language understanding, reasoning, and generation, with variants ranging from 0.3 billion to 424 billion parameters. The models incorporate both dense and Mixture-of-Experts (MoE) architectures, with the largest MoE models efficiently scaling parameters by activating only a subset of experts per token, thus balancing model capacity and computational efficiency. Built on Baidus multi-stage pretraining pipeline, ERNIE 4.5 models are trained on a vast corpus of 5