MOPD is a post-training method that combines multiple RL-trained domain teachers into one model via on-policy distillation. It reduces exposure bias, improves efficiency, and outperforms existing multi-domain training methods on Qwen3-30B-A3B. It has also been applied in MiMo-V2-Flash at industrial scale.</p>\n","updatedAt":"2026-07-01T13:51:38.068Z","author":{"_id":"63a369d98c0c89dcae3b8329","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63a369d98c0c89dcae3b8329/AiH2zjy1cnt9OADAAZMLD.jpeg","fullname":"Adina Yakefu","name":"AdinaY","type":"user","isPro":false,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":1182,"isUserFollowing":false,"primaryOrg":{"avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1583856921041-5dd96eb166059660ed1ee413.png","fullname":"Hugging Face","name":"huggingface","type":"org","isHf":true,"details":"The AI community building the future.","plan":"team"}}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9332396388053894},"editors":["AdinaY"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/63a369d98c0c89dcae3b8329/AiH2zjy1cnt9OADAAZMLD.jpeg"],"reactions":[],"isReport":false}},{"id":"6a45c3b1456c75f1f5af2265","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":372,"isUserFollowing":false},"createdAt":"2026-07-02T01:49:37.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training](https://huggingface.co/papers/2605.12483) (2026)\n* [Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence](https://huggingface.co/papers/2605.13230) (2026)\n* [A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillation](https://huggingface.co/papers/2605.12227) (2026)\n* [Stabilizing On-Policy Distillation for MLLM Reasoning with Global Normalization](https://huggingface.co/papers/2606.09091) (2026)\n* [Trust Region On-Policy Distillation](https://huggingface.co/papers/2606.01249) (2026)\n* [SOD: Step-wise On-policy Distillation for Small Language Model Agents](https://huggingface.co/papers/2605.07725) (2026)\n* [Trajectory-Refined Distillation](https://huggingface.co/papers/2606.08432) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"<p>This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2605.12483\">Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.13230\">Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.12227\">A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillation</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.09091\">Stabilizing On-Policy Distillation for MLLM Reasoning with Global Normalization</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.01249\">Trust Region On-Policy Distillation</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.07725\">SOD: Step-wise On-policy Distillation for Small Language Model Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.08432\">Trajectory-Refined Distillation</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code>@librarian-bot recommend</code></p>\n","updatedAt":"2026-07-02T01:49:37.335Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":372,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7371894121170044},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.30406","authors":[{"_id":"6a451b254f1dd35e48fb8ce0","user":{"_id":"6686f695840ee769597de318","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6686f695840ee769597de318/50XZQ80Y2PElI35O0qVFc.jpeg","isPro":false,"fullname":"Wenhan Ma","user":"CuteNPC","type":"user","name":"CuteNPC"},"name":"Wenhan Ma","status":"admin_assigned","statusLastChangedAt":"2026-07-01T13:53:56.968Z","hidden":false},{"_id":"6a451b254f1dd35e48fb8ce1","name":"Jianyu Wei","hidden":false},{"_id":"6a451b254f1dd35e48fb8ce2","name":"Liang Zhao","hidden":false},{"_id":"6a451b254f1dd35e48fb8ce3","name":"Hailin Zhang","hidden":false},{"_id":"6a451b254f1dd35e48fb8ce4","name":"Bangjun Xiao","hidden":false},{"_id":"6a451b254f1dd35e48fb8ce5","name":"Lei Li","hidden":false},{"_id":"6a451b254f1dd35e48fb8ce6","name":"Qibin Yang","hidden":false},{"_id":"6a451b254f1dd35e48fb8ce7","name":"Bofei Gao","hidden":false},{"_id":"6a451b254f1dd35e48fb8ce8","name":"Yudong Wang","hidden":false},{"_id":"6a451b254f1dd35e48fb8ce9","name":"Rang Li","hidden":false},{"_id":"6a451b254f1dd35e48fb8cea","name":"Jinhao Dong","hidden":false},{"_id":"6a451b254f1dd35e48fb8ceb","name":"Zhifang Sui","hidden":false},{"_id":"6a451b254f1dd35e48fb8cec","user":{"_id":"6538815d1bdb3c40db94fbfa","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6538815d1bdb3c40db94fbfa/id7aSY8JUgKK2agKWLERt.jpeg","isPro":false,"fullname":"Fuli Luo","user":"luofuli","type":"user","name":"luofuli"},"name":"Fuli Luo","status":"admin_assigned","statusLastChangedAt":"2026-07-01T13:53:26.460Z","hidden":false}],"publishedAt":"2026-06-29T00:00:00.000Z","submittedOnDailyAt":"2026-07-01T00:00:00.000Z","title":"MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training","submittedOnDailyBy":{"_id":"63a369d98c0c89dcae3b8329","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63a369d98c0c89dcae3b8329/AiH2zjy1cnt9OADAAZMLD.jpeg","isPro":false,"fullname":"Adina Yakefu","user":"AdinaY","type":"user","name":"AdinaY"},"summary":"Modern large language models (LLMs) rely on reinforcement learning during post-training to push specific capabilities, yet integrating multiple capabilities into one model remains hard. Existing methods, such as Off-Policy Finetune and Mix-RL, are either inefficient or lose performance. In this work, we propose Multi-teacher On-Policy Distillation (MOPD), a post-training paradigm for combining the capabilities of multiple domain RL teachers: we first run per-domain specialised RL to obtain a set of domain teachers, then distill these teachers into the student on its own rollouts. This eliminates exposure bias and provides a dense optimization signal. On Qwen3-30B-A3B, MOPD outperforms Mix-RL, Cascade RL, Off-Policy Finetune, and Param-Merge baselines, inheriting nearly all of each teacher's capability. MOPD also enables parallel, independent development of domain teachers, removing the cross-domain coupling typical of multi-domain post-training. MOPD has been deployed in the post-training of MiMo-V2-Flash, an industrial-scale frontier model, demonstrating its practical value for capability integration in frontier-scale LLMs.","upvotes":4,"discussionId":"6a451b264f1dd35e48fb8ced","ai_summary":"Multi-teacher On-Policy Distillation (MOPD) enables efficient integration of multiple domain capabilities in large language models through specialized reinforcement learning teachers and on-policy distillation, achieving superior performance over existing methods.","ai_keywords":["reinforcement learning","post-training","domain RL teachers","on-policy distillation","exposure bias","dense optimization signal","Qwen3-30B-A3B","Mix-RL","Cascade RL","Off-Policy Finetune","Param-Merge","MiMo-V2-Flash"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","organization":{"_id":"680cb4c37f289defb2210940","name":"XiaomiMiMo","fullname":"Xiaomi MiMo","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/680cb7d1233834890a64acee/5w_4aLfF-7MAyaIPOV498.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"},{"_id":"661ab1f1fa3b144a381fa454","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/661ab1f1fa3b144a381fa454/IlpZBb9NCjo7ntFwMIH53.png","isPro":false,"fullname":"Urro","user":"urroxyz","type":"user"},{"_id":"651c80a26ba9ab9b9582c273","avatarUrl":"/avatars/e963452eafd21f517d800f2e58e0f918.svg","isPro":false,"fullname":"siyeng feng","user":"siyengfeng","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"680cb4c37f289defb2210940","name":"XiaomiMiMo","fullname":"Xiaomi MiMo","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/680cb7d1233834890a64acee/5w_4aLfF-7MAyaIPOV498.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.30406.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2606.30406 in a model README.md to link it from this page.

Cite arxiv.org/abs/2606.30406 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2606.30406 in a Space README.md to link it from this page.