Isn't the core framework here exactly the same as Orthrus, released 4 months ago? <a href=\"https://arxiv.org/abs/2605.12825\" rel=\"nofollow\">https://arxiv.org/abs/2605.12825</a></p>\n","updatedAt":"2026-09-04T18:40:26.137Z","author":{"_id":"63edf17df5aef7e657262605","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63edf17df5aef7e657262605/X3WzygIyND_0T_SWRDDU4.jpeg","fullname":"Nguyen Van Chien","name":"chiennv","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":55,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8811411261558533},"editors":["chiennv"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/63edf17df5aef7e657262605/X3WzygIyND_0T_SWRDDU4.jpeg"],"reactions":[{"reaction":"🤯","users":["huy-nh-2000"],"count":1}],"isReport":false}},{"id":"6a9bb33174b3fb3c35b2240b","author":{"_id":"661839d73b412cdc851299c1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png","fullname":"Subham Sekhar Sahoo","name":"s-sahoo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":14,"isUserFollowing":false},"createdAt":"2026-09-05T06:14:09.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"Thanks for brining this to our attention.\n\nThe core idea of our method Uno is to achieve AR and diffusion generation keeping the architecture (with its attention) unchanged, but the paper that you shared changes the architecture by adding diffusion attention heads.\n\nNot only that, it uses bidirectional attention for the diffusion blocks.\n\nThere are a lot of similarities, yes, but i respectfully disagree with this characterization that the core ideas are the same.\n\nWe’ll add a discussion in the next revision","html":"<p>Thanks for brining this to our attention.</p>\n<p>The core idea of our method Uno is to achieve AR and diffusion generation keeping the architecture (with its attention) unchanged, but the paper that you shared changes the architecture by adding diffusion attention heads.</p>\n<p>Not only that, it uses bidirectional attention for the diffusion blocks.</p>\n<p>There are a lot of similarities, yes, but i respectfully disagree with this characterization that the core ideas are the same.</p>\n<p>We’ll add a discussion in the next revision</p>\n","updatedAt":"2026-09-05T06:14:35.322Z","author":{"_id":"661839d73b412cdc851299c1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png","fullname":"Subham Sekhar Sahoo","name":"s-sahoo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":14,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.9403149485588074},"editors":["s-sahoo"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png"],"reactions":[],"isReport":false},"replies":[{"id":"6a9c57a098a57b458c728208","author":{"_id":"63edf17df5aef7e657262605","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63edf17df5aef7e657262605/X3WzygIyND_0T_SWRDDU4.jpeg","fullname":"Nguyen Van Chien","name":"chiennv","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":55,"isUserFollowing":false},"createdAt":"2026-09-05T17:55:44.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"Thanks. I am referring to the core framework, not the exact implementation. I don't think \"unchanged\" vs. \"changed\" architecture is an accurate distinction: both preserve a frozen Transformer backbone and augment it with trainable diffusion parameters—LoRA in Uno and separate attention in Orthrus. Also, causal vs bidirectional attention is just simply a design choice within the diffusion path. \n\nAt the framework level: keep the AR weights frozen, add and train a diffusion pathway, reuse the AR KV cache, draft multiple tokens in parallel, and verify them with the AR weights for lossless generation. That is the same core framework. \nYour claim that Uno is the first diffusion-based model to achieve lossless generation without a separate drafter is not correct.","html":"<p>Thanks. I am referring to the core framework, not the exact implementation. I don't think \"unchanged\" vs. \"changed\" architecture is an accurate distinction: both preserve a frozen Transformer backbone and augment it with trainable diffusion parameters—LoRA in Uno and separate attention in Orthrus. Also, causal vs bidirectional attention is just simply a design choice within the diffusion path. </p>\n<p>At the framework level: keep the AR weights frozen, add and train a diffusion pathway, reuse the AR KV cache, draft multiple tokens in parallel, and verify them with the AR weights for lossless generation. That is the same core framework.<br>Your claim that Uno is the first diffusion-based model to achieve lossless generation without a separate drafter is not correct.</p>\n","updatedAt":"2026-09-05T17:57:01.359Z","author":{"_id":"63edf17df5aef7e657262605","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63edf17df5aef7e657262605/X3WzygIyND_0T_SWRDDU4.jpeg","fullname":"Nguyen Van Chien","name":"chiennv","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":55,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.8694643378257751},"editors":["chiennv"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/63edf17df5aef7e657262605/X3WzygIyND_0T_SWRDDU4.jpeg"],"reactions":[],"isReport":false,"parentCommentId":"6a9bb33174b3fb3c35b2240b"}},{"id":"6a9edd965534553f7c895eb7","author":{"_id":"661839d73b412cdc851299c1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png","fullname":"Subham Sekhar Sahoo","name":"s-sahoo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":14,"isUserFollowing":false},"createdAt":"2026-09-07T15:51:50.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"Unlike your approach, the core idea is to use the `AR` architecture for both drafting and verification. **Your approach modifies the architecture** during the drafting phase by introducing diffusion heads.","html":"<p>Unlike your approach, the core idea is to use the <code>AR</code> architecture for both drafting and verification. <strong>Your approach modifies the architecture</strong> during the drafting phase by introducing diffusion heads.</p>\n","updatedAt":"2026-09-07T15:52:19.608Z","author":{"_id":"661839d73b412cdc851299c1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png","fullname":"Subham Sekhar Sahoo","name":"s-sahoo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":14,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.9169875383377075},"editors":["s-sahoo"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png"],"reactions":[],"isReport":false,"parentCommentId":"6a9bb33174b3fb3c35b2240b"}}]},{"id":"6a9fbcb93c7ddcb8521cce69","author":{"_id":"661839d73b412cdc851299c1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png","fullname":"Subham Sekhar Sahoo","name":"s-sahoo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":14,"isUserFollowing":false},"createdAt":"2026-09-08T07:43:53.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Diffusion LLMs have two major issues wrt AR LLMs:\n1. Poor quality,\n2. Slow inference at high batch sizes.\n\nTo overcome these limitations, we introduce `diffusion-augmented LLMs`, a new class\nof models that defines an AR model distribution while using diffusion to draw multiple\ntokens in parallel from that distribution. We decouple the parameters of these models into\ntwo sets: \n- **AR weights**: Trained using next-token prediction loss to define an AR distribution.\n- **Diffusion weights**: Trained to generate multiple tokens in parallel from the AR distribution.\nunder a single unified LLM architecture. \n\nWe name our model `Uno`. To sample from it, we propose **$\\Psi$-Spec** sampler, which enables provably lossless multi-token prediction from the AR distribution.\n- Unlike speculative decoding methods, `Uno` does not require a separately trained draft model and achieves higher throughput across all batch sizes.\n- Unlike self-speculative decoding approaches, `Uno` is lossless. \n\n## Results\nOur model `Uno` \n- Achieves higher throughput than speculative-decoding methods at every batch size, and \n- Delivers up to 3× speedups over the base AR model, including at the largest batch size supported by the\ndevice. \n\nNotably, our 8B Uno model outperforms the leading open d-LLM, the `26B DiffusionGemma`, and the proprietary Mercury 2 across all evaluated benchmarks in agentic tool use, coding, and long-context reasoning.","html":"<p>Diffusion LLMs have two major issues wrt AR LLMs:</p>\n<ol>\n<li>Poor quality,</li>\n<li>Slow inference at high batch sizes.</li>\n</ol>\n<p>To overcome these limitations, we introduce <code>diffusion-augmented LLMs</code>, a new class<br>of models that defines an AR model distribution while using diffusion to draw multiple<br>tokens in parallel from that distribution. We decouple the parameters of these models into<br>two sets: </p>\n<ul>\n<li><strong>AR weights</strong>: Trained using next-token prediction loss to define an AR distribution.</li>\n<li><strong>Diffusion weights</strong>: Trained to generate multiple tokens in parallel from the AR distribution.<br>under a single unified LLM architecture.</li>\n</ul>\n<p>We name our model <code>Uno</code>. To sample from it, we propose <strong>$\\Psi$-Spec</strong> sampler, which enables provably lossless multi-token prediction from the AR distribution.</p>\n<ul>\n<li>Unlike speculative decoding methods, <code>Uno</code> does not require a separately trained draft model and achieves higher throughput across all batch sizes.</li>\n<li>Unlike self-speculative decoding approaches, <code>Uno</code> is lossless.</li>\n</ul>\n<h2 class=\"relative group flex items-baseline\">\n\t<a id=\"results\" class=\"block pr-1.5 text-lg md:absolute md:p-1.5 md:opacity-0 md:group-hover:opacity-100 md:right-full\" href=\"#results\" rel=\"nofollow\">\n\t\t<span class=\"header-link\"><svg class=\"text-gray-500 hover:text-black dark:hover:text-gray-200 w-4\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" aria-hidden=\"true\" role=\"img\" width=\"1em\" height=\"1em\" preserveAspectRatio=\"xMidYMid meet\" viewBox=\"0 0 256 256\"><path d=\"M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z\" fill=\"currentColor\"></path></svg></span>\n\t</a>\n\t<span>\n\t\tResults\n\t</span>\n</h2>\n<p>Our model <code>Uno</code> </p>\n<ul>\n<li>Achieves higher throughput than speculative-decoding methods at every batch size, and </li>\n<li>Delivers up to 3× speedups over the base AR model, including at the largest batch size supported by the<br>device.</li>\n</ul>\n<p>Notably, our 8B Uno model outperforms the leading open d-LLM, the <code>26B DiffusionGemma</code>, and the proprietary Mercury 2 across all evaluated benchmarks in agentic tool use, coding, and long-context reasoning.</p>\n","updatedAt":"2026-09-08T07:43:53.074Z","author":{"_id":"661839d73b412cdc851299c1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png","fullname":"Subham Sekhar Sahoo","name":"s-sahoo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":14,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.849663257598877},"editors":["s-sahoo"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2609.04010","authors":[{"_id":"6a9a54878f7c3b7557239667","name":"Subham Sekhar Sahoo","hidden":false},{"_id":"6a9a54878f7c3b7557239668","name":"Lingjie Chen","hidden":false},{"_id":"6a9a54878f7c3b7557239669","name":"Khiem Pham","hidden":false},{"_id":"6a9a54878f7c3b755723966a","name":"Jonathan Geuter","hidden":false},{"_id":"6a9a54878f7c3b755723966b","name":"Chaitanya Dwivedi","hidden":false},{"_id":"6a9a54878f7c3b755723966c","name":"Varad Pimpalkhute","hidden":false},{"_id":"6a9a54878f7c3b755723966d","name":"Yash Akhauri","hidden":false},{"_id":"6a9a54878f7c3b755723966e","name":"Alexander Moreno","hidden":false},{"_id":"6a9a54878f7c3b755723966f","user":{"_id":"6552e5121c6e798ec0d3d52c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6552e5121c6e798ec0d3d52c/y30R447PkY2F1_T4HuyMp.jpeg","isPro":false,"fullname":"Mikhail Yurochkin","user":"moonfolk","type":"user","name":"moonfolk"},"name":"Mikhail Yurochkin","status":"claimed_verified","statusLastChangedAt":"2026-09-08T08:45:04.297Z","hidden":false},{"_id":"6a9a54878f7c3b7557239670","name":"Zhenting Wang","hidden":false},{"_id":"6a9a54878f7c3b7557239671","user":{"_id":"63c9725ebedad7e2bf160bdc","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63c9725ebedad7e2bf160bdc/wzPuyhOXCYBNGwZDshbnL.jpeg","isPro":false,"fullname":"Mostafa Elhoushi","user":"melhoushi","type":"user","name":"melhoushi"},"name":"Mostafa Elhoushi","status":"claimed_verified","statusLastChangedAt":"2026-09-07T08:45:04.200Z","hidden":false},{"_id":"6a9a54878f7c3b7557239672","name":"Nolan Dey","hidden":false},{"_id":"6a9a54878f7c3b7557239673","name":"Shane Bergsma","hidden":false},{"_id":"6a9a54878f7c3b7557239674","name":"Joel Hestness","hidden":false},{"_id":"6a9a54878f7c3b7557239675","name":"John Thickstun","hidden":false},{"_id":"6a9a54878f7c3b7557239676","name":"Eric Xing","hidden":false},{"_id":"6a9a54878f7c3b7557239677","name":"Zhengzhong Liu","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/661839d73b412cdc851299c1/4V1uvWZASfgGD6y-bq9s0.png","https://cdn-uploads.huggingface.co/production/uploads/661839d73b412cdc851299c1/9d3KThaaMf_dzCN6gy3FP.png"],"publishedAt":"2026-09-03T00:00:00.000Z","submittedOnDailyAt":"2026-09-08T00:00:00.000Z","title":"Unlocking Lossless Speedups in LLMs via Discrete Diffusion","submittedOnDailyBy":{"_id":"661839d73b412cdc851299c1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png","isPro":false,"fullname":"Subham Sekhar Sahoo","user":"s-sahoo","type":"user","name":"s-sahoo"},"summary":"Large Language Models (LLMs) owe much of their success to next-token prediction (NTP), but their autoregressive (AR) structure requires slow, sequential token generation. To overcome this bottleneck, we introduce diffusion-augmented LLMs, a new class of models that defines an AR model distribution while using diffusion to draw multiple tokens in parallel from that distribution. We decouple the parameters of these models into two sets: AR weights, trained using the standard NTP objective, and lightweight diffusion weights, trained to generate multiple tokens simultaneously. The diffusion weights are learned through a simple Diffusion Distillation phase that adds negligible overhead to existing LLM training pipelines. We also introduce Ψ-Spec, a family of samplers that enables lossless acceleration and inference-time scaling at a fixed context length. Unlike speculative decoding, our method requires no separate draft model. Unlike diffusion LLMs (d-LLMs), it accelerates generation without sacrificing the quality of the underlying AR model. The resulting models, called Uno, can be trained from scratch or built by augmenting existing open-weight AR LLMs. Uno achieves higher throughput than leading speculative-decoding methods at every evaluated batch size and delivers up to 3times speedups over the base AR model, including at the largest batch size supported by the device. Notably, our 8B Uno model outperforms the leading open d-LLM, the 26B DiffusionGemma, and the proprietary Mercury 2 across all evaluated benchmarks in agentic tool use, coding, and long-context reasoning. We release code and checkpoints at: https://s-sahoo.github.io/uno/","upvotes":19,"discussionId":"6a9a54888f7c3b7557239678","projectPage":"https://s-sahoo.com/uno/","githubRepo":"https://github.com/ifm-ai/uno","githubRepoAddedBy":"user","ai_summary":"Diffusion-augmented autoregressive language models use parallel token sampling via distilled diffusion weights and a specialized sampler to accelerate inference without quality loss or draft models.","ai_keywords":["next-token prediction","autoregressive","diffusion-augmented LLMs","diffusion distillation","Ψ-Spec","speculative decoding","diffusion LLMs","Uno"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":44,"organization":{"_id":"654bd1effb8732d0553a2a5b","name":"IFM","fullname":"Institute of Foundation Models","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/69e0929c5731872fd66f3b3c/AHedSSAaCV5S09amCYaWF.webp"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"670f8d8801f1ec4d87e3f1f2","avatarUrl":"/avatars/4afd117d451866aa23ec9fb8e46928b3.svg","isPro":false,"fullname":"Aaryamonvikram Singh","user":"aaryamonvikram","type":"user"},{"_id":"661839d73b412cdc851299c1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/661839d73b412cdc851299c1/xicwANPQPTFdWfblisL2-.png","isPro":false,"fullname":"Subham Sekhar Sahoo","user":"s-sahoo","type":"user"},{"_id":"63c9725ebedad7e2bf160bdc","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63c9725ebedad7e2bf160bdc/wzPuyhOXCYBNGwZDshbnL.jpeg","isPro":false,"fullname":"Mostafa Elhoushi","user":"melhoushi","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"64dfcc62e8b6f3f3baa950e0","avatarUrl":"/avatars/21bbff67d46c08044efe2406575aa77e.svg","isPro":false,"fullname":"Zhenting Wang","user":"ztwang","type":"user"},{"_id":"65025370b6595dc45c397340","avatarUrl":"/avatars/9469599b176034548042922c0afa7051.svg","isPro":false,"fullname":"J C","user":"dark-pen","type":"user"},{"_id":"687016ec08927be2e4413d6d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/lKP9PoghnhdtMgz4M60TF.png","isPro":true,"fullname":"Haoran Yuan","user":"JensenYuan","type":"user"},{"_id":"64ea6965b96ff0e1753c128a","avatarUrl":"/avatars/4763d3e76b10cfb82e09fb0c26875efb.svg","isPro":false,"fullname":"Amr Hegazy","user":"Amr-Hegazy","type":"user"},{"_id":"6552e5121c6e798ec0d3d52c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6552e5121c6e798ec0d3d52c/y30R447PkY2F1_T4HuyMp.jpeg","isPro":false,"fullname":"Mikhail Yurochkin","user":"moonfolk","type":"user"},{"_id":"66448b4fecac3bc79b26304f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66448b4fecac3bc79b26304f/aPH3UFbc20CL2Bz2yn7nH.jpeg","isPro":false,"fullname":"Hadi Abdine","user":"habdine","type":"user"},{"_id":"673b163b7c1c0ded76b08287","avatarUrl":"/avatars/b46678de1ed1f39f41181ac6347f0e6a.svg","isPro":false,"fullname":"Mohammad Areeb Qazi","user":"mohammadareebqazi","type":"user"},{"_id":"678bf205c84e7bb993bedad4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/FJ0kc2YrvmL0crTH-Y0QD.png","isPro":false,"fullname":"Owais Aijaz","user":"owees17","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":1,"organization":{"_id":"654bd1effb8732d0553a2a5b","name":"IFM","fullname":"Institute of Foundation Models","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/69e0929c5731872fd66f3b3c/AHedSSAaCV5S09amCYaWF.webp"},"query":{}}">