We study why a 2B model fails at interactive dialogue games, and find the failures are not only missing knowledge but local decision errors — repeated guesses, malformed actions, violations of feedback the model just saw. Since these are mechanically checkable, we place supervision exactly there: broad SFT to acquire game participation, turn-local DPO pairs (same history, one corrupted decision) to repair diagnosed failures in a single game family, and training-free scaling of the learned LoRA delta to preserve general capability.</p>\n<p>Official final evaluation: public clemscore 10.67 → 38.92, closed in-domain 13.41 → 41.17 (above the 4B and 9B baselines), with static performance approximately preserved (44.14 vs. 44.24). Out-of-domain remains low at 7.88 — transfer is concentrated within the targeted family.</p>\n<p>Model: <a href=\"https://huggingface.co/chnln/Qwen3.5-2B-playpen-playornotplay\">https://huggingface.co/chnln/Qwen3.5-2B-playpen-playornotplay</a></p>\n","updatedAt":"2026-08-31T19:40:02.488Z","author":{"_id":"61bf8017c88f3fd22f654086","avatarUrl":"/avatars/ea9e762b3db755fe051751577353fbca.svg","fullname":"Nan Li","name":"chnln","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.911110520362854},"editors":["chnln"],"editorAvatarUrls":["/avatars/ea9e762b3db755fe051751577353fbca.svg"],"reactions":[],"isReport":false}},{"id":"6a962bff536d0ff62b62a40d","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":379,"isUserFollowing":false},"createdAt":"2026-09-01T01:35:59.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Toward Anthropomorphic Dialogue: A Closed-Loop Framework for Human-Like Chat Generation, Evaluation, and Preference Alignment](https://huggingface.co/papers/2607.17191) (2026)\n* [Wnuan: Staged Post-Training for Question Answering over Proprietary Enterprise Knowledge](https://huggingface.co/papers/2608.01862) (2026)\n* [Cross-Benchmark Generalization in Long-Horizon Agents](https://huggingface.co/papers/2608.00181) (2026)\n* [Search, Fail, Recover: A Training Framework for Correction-Aware Reasoning](https://huggingface.co/papers/2607.07492) (2026)\n* [Inverse RL Helps Align AI by Imitating Humans](https://huggingface.co/papers/2607.24900) (2026)\n* [The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning](https://huggingface.co/papers/2607.20952) (2026)\n* [When Reasoning Narrows the Move: Diversity Collapse in LLM Game Play](https://huggingface.co/papers/2607.19523) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"<p>This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2607.17191\">Toward Anthropomorphic Dialogue: A Closed-Loop Framework for Human-Like Chat Generation, Evaluation, and Preference Alignment</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2608.01862\">Wnuan: Staged Post-Training for Question Answering over Proprietary Enterprise Knowledge</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2608.00181\">Cross-Benchmark Generalization in Long-Horizon Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.07492\">Search, Fail, Recover: A Training Framework for Correction-Aware Reasoning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.24900\">Inverse RL Helps Align AI by Imitating Humans</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.20952\">The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.19523\">When Reasoning Narrows the Move: Diversity Collapse in LLM Game Play</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code>@librarian-bot recommend</code></p>\n","updatedAt":"2026-09-01T01:35:59.165Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":379,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7436875700950623},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.28458","authors":[{"_id":"6a94d7f8073195fee51571c0","name":"Nan Li","hidden":false}],"publishedAt":"2026-08-28T00:00:00.000Z","submittedOnDailyAt":"2026-08-31T00:00:00.000Z","title":"Acquire, Repair, Preserve: A Diagnosis-Guided Post-Training Recipe for Small-Model Dialogue Game Agents","submittedOnDailyBy":{"_id":"61bf8017c88f3fd22f654086","avatarUrl":"/avatars/ea9e762b3db755fe051751577353fbca.svg","isPro":false,"fullname":"Nan Li","user":"chnln","type":"user","name":"chnln"},"summary":"Interactive dialogue games test a capability that static benchmarks largely leave implicit: a model must carry state across turns, interpret feedback, and choose valid actions under changing constraints. We study this setting in the LM Playschool Challenge with a 2B open-weight model, and find that many failures are not only broad knowledge failures but also local decision failures: repeated guesses, malformed actions, and violations of feedback that the model has just seen. These diagnostics motivate a training recipe organized around three steps: acquire broad game participation through supervised fine-tuning, repair mechanically verifiable failures within one targeted dialogue-game family using turn-local preference pairs, and preserve general capabilities beyond these dialogue games. In the official final evaluation, our submission improves public clemscore from 10.67 to 38.92 and closed in-domain score from 13.41 to 41.17, while approximately preserving aggregate static performance (44.14 vs. 44.24 for the baseline). Out-of-domain clemscore remains low at 7.88, with the largest gains concentrated in unseen variants of the targeted family. Our results suggest that broad SFT brings most of the model's capability improvement; turn-local supervision can be effective when failure detection is precise, with observed transfer concentrated primarily within-family.","upvotes":2,"discussionId":"6a94d7f8073195fee51571c1","ai_summary":"A 2B open-weight model improves interactive dialogue-game performance through supervised fine-tuning and turn-local preference training, though gains remain largely within the targeted game family.","ai_keywords":["supervised fine-tuning","preference pairs","turn-local supervision","dialogue games","state tracking","feedback interpretation"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"67110a86b798bde20c43ac84","name":"cs-nlp-uu","fullname":"NLP Group at Utrecht University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6305d542435ec751b72434b8/A9YelDlLilyEoGkqaksoL.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"61bf8017c88f3fd22f654086","avatarUrl":"/avatars/ea9e762b3db755fe051751577353fbca.svg","isPro":false,"fullname":"Nan Li","user":"chnln","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"67110a86b798bde20c43ac84","name":"cs-nlp-uu","fullname":"NLP Group at Utrecht University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6305d542435ec751b72434b8/A9YelDlLilyEoGkqaksoL.png"},"query":{}}">
Acquire, Repair, Preserve: A Diagnosis-Guided Post-Training Recipe for Small-Model Dialogue Game Agents
Published on Aug 28
· Submitted by Nan Li on Aug 31 Abstract
A 2B open-weight model improves interactive dialogue-game performance through supervised fine-tuning and turn-local preference training, though gains remain largely within the targeted game family.
Interactive dialogue games test a capability that static benchmarks largely leave implicit: a model must carry state across turns, interpret feedback, and choose valid actions under changing constraints. We study this setting in the LM Playschool Challenge with a 2B open-weight model, and find that many failures are not only broad knowledge failures but also local decision failures: repeated guesses, malformed actions, and violations of feedback that the model has just seen. These diagnostics motivate a training recipe organized around three steps: acquire broad game participation through supervised fine-tuning, repair mechanically verifiable failures within one targeted dialogue-game family using turn-local preference pairs, and preserve general capabilities beyond these dialogue games. In the official final evaluation, our submission improves public clemscore from 10.67 to 38.92 and closed in-domain score from 13.41 to 41.17, while approximately preserving aggregate static performance (44.14 vs. 44.24 for the baseline). Out-of-domain clemscore remains low at 7.88, with the largest gains concentrated in unseen variants of the targeted family. Our results suggest that broad SFT brings most of the model's capability improvement; turn-local supervision can be effective when failure detection is precise, with observed transfer concentrated primarily within-family.
Community
We study why a 2B model fails at interactive dialogue games, and find the failures are not only missing knowledge but local decision errors — repeated guesses, malformed actions, violations of feedback the model just saw. Since these are mechanically checkable, we place supervision exactly there: broad SFT to acquire game participation, turn-local DPO pairs (same history, one corrupted decision) to repair diagnosed failures in a single game family, and training-free scaling of the learned LoRA delta to preserve general capability.
Official final evaluation: public clemscore 10.67 → 38.92, closed in-domain 13.41 → 41.17 (above the 4B and 9B baselines), with static performance approximately preserved (44.14 vs. 44.24). Out-of-domain remains low at 7.88 — transfer is concentrated within the targeted family.
Model: https://huggingface.co/chnln/Qwen3.5-2B-playpen-playornotplay
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.28458 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.28458 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.28458 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.