Alibaba Qwen Unveils Qwen3-4B-Instruct-2507 and Qwen3-4B-Thinking-2507: Refreshing the Importance of Small Language Models

Alibabas Qwen team has introduced Qwen3-4B-Instruct-2507 and Qwen3-4B-Thinking-2507, two compact yet highly capable language models with only 4 billion parameters that excel across general and expert tasks while operating efficiently on consumer hardware. These models feature a native 256K token context window, enabling them to process extremely long inputs such as large codebases, multi-document archives, and extended dialogues without external modifications, marking a significant advancement in long-context AI capabilities. Built with 36 transformer layers and utilizing Grouped Query Attention (GQA)