ByteDance Researchers Introduce Seed-Coder: A Model-Centric Code LLM Trained on 6 Trillion Tokens

ByteDance's researchers have developed Seed-Coder, an open-source family of 8-billion-parameter language models that significantly reduce human intervention in code data curation by employing a model-centric pipeline. This innovative approach leverages large language models to automatically score and filter vast code datasets from sources like GitHub, culminating in a 6-trillion-token dataset that enhances the model's coding and reasoning capabilities. Unlike traditional methods reliant on manual filtering and expert rules, Seed-Coder's pipeline emphasizes scalability and data-driven processes, aligning with the broader trend that breakthroughs in AI stem from large-scale, automated data collection