snkii/Sori-1B: Audio-Grounded LM Trained From Scratch (No Text-Only Pretraining)

Sori-1B is a 1B-parameter audio-language model built by a single SNU researcher whose core claim to fame is that its decoder is trained entirely from scratch on audio-paired text — no text-only pretraining, no pretrained-LM initialization — with the idea being that a language model which has only ever seen text alongside sound will actually ground its answers in audio rather than leaning on text-only priors like typical AF3-style models do (they show AF3 retains ~74% of its above-chance MMAU margin even when the audio is replaced with silence).

It reuses NVIDIA’s frozen Audio Flamingo Next encoder (61.5% of params, untouched) while everything else — decoder, embeddings, output head, a custom “auditory-ontology” tokenizer built from audio concept categories instead of text-derived vocab — is trained from scratch on ~7.4k hours / 4.75M examples using just 3x RTX 4090s. It supports MCQ, open QA, captioning, and ASR modes, ships an inference endpoint handler and a synthetic-audio terminal demo, and includes an MMAU test-mini eval script, but the weights are gated under a non-commercial/academic-only license (since it redistributes NVIDIA’s encoder under NVIDIA’s OneWay Noncommercial terms) and the repo itself is marked “coming soon.”