AntLing’ve open-sourced 6 Base Model checkpoints for Ling-3.0-tiny & Ling-3.0-flash, covering pre-trained, mid-trained, and WSM-merged stages.

None has undergone post-training, giving researchers flexible starting points for continued pre-training, fine-tuning, and further research.

Two key highlights:
- They use WSM to replace LR decay with weighted checkpoint merging, making the training process better suited for continual pre-training while enabling offline exploration of different LR decay strategies.
- With one shared training recipe, the community can validate strategies on tiny-base, then scale them to flash-base.

#1- Ling-3.0-tiny-base: 7.9B total | 1.3B active.

Despite having only half as many total parameters as Ling-2.5-mini-base, Ling-3.0-tiny-base delivers comparable or superior performance on most benchmarks, with particularly strong results in coding.
For code pre-training/SFT, RL post-training, teaching, model behavior & MoE studies.

https://preview.redd.it/8edbwdc6tckh1.png?width=900&format=png&auto=webp&s=21de82bdad283fc897e9753ce6bef753e69818f9

#2- Ling-3.0-flash-base: 124B total | 5.1B active.

In evaluations, Ling-3.0-flash-base achieves strong performance across coding, reasoning, and long context tasks, even when compared with models 2 to 3 times larger.
This makes it well suited for continued pre-training, post training, and domain adaptation in coding, long horizon workflows, finance, healthcare, and other specialized applications.

https://preview.redd.it/q3kueuj9tckh1.png?width=1199&format=png&auto=webp&s=1d8e27d09068fb200a82b57ecf6bef10260c967e