We are happy to announce <strong>Lego-RL: an open-source framework for harness-native reinforcement learning of coding agents</strong>. All code, data, and models are available on GitHub and Hugging Face.</p>\n","updatedAt":"2026-08-19T16:52:23.538Z","author":{"_id":"63c20105726f62e411fbe882","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63c20105726f62e411fbe882/2UsU9O2psbDjJzz-sAmGH.jpeg","fullname":"Yuxin Jiang","name":"YuxinJiang","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.8860148191452026},"editors":["YuxinJiang"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/63c20105726f62e411fbe882/2UsU9O2psbDjJzz-sAmGH.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.17393","authors":[{"_id":"6a855dd4536bdd3bdd48f92a","name":"Yiming Du","hidden":false},{"_id":"6a855dd4536bdd3bdd48f92b","name":"Yuxin Jiang","hidden":false},{"_id":"6a855dd4536bdd3bdd48f92c","name":"Tao Yuan","hidden":false},{"_id":"6a855dd4536bdd3bdd48f92d","name":"Jianbo Dai","hidden":false},{"_id":"6a855dd4536bdd3bdd48f92e","name":"Shaowei Wang","hidden":false},{"_id":"6a855dd4536bdd3bdd48f92f","name":"Jierun Chen","hidden":false},{"_id":"6a855dd4536bdd3bdd48f930","name":"Chaofan Tao","hidden":false},{"_id":"6a855dd4536bdd3bdd48f931","name":"Xianzhi Yu","hidden":false},{"_id":"6a855dd4536bdd3bdd48f932","name":"Lifeng Shang","hidden":false},{"_id":"6a855dd4536bdd3bdd48f933","name":"Kam-Fai Wong","hidden":false},{"_id":"6a855dd4536bdd3bdd48f934","name":"Xiaohui Li","hidden":false},{"_id":"6a855dd4536bdd3bdd48f935","name":"Haoli Bai","hidden":false}],"publishedAt":"2026-08-18T00:00:00.000Z","submittedOnDailyAt":"2026-08-19T00:00:00.000Z","title":"LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents","submittedOnDailyBy":{"_id":"63c20105726f62e411fbe882","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63c20105726f62e411fbe882/2UsU9O2psbDjJzz-sAmGH.jpeg","isPro":false,"fullname":"Yuxin Jiang","user":"YuxinJiang","type":"user","name":"YuxinJiang"},"summary":"Reinforcement learning for coding agents increasingly relies on long-running agent harnesses to manage tool integration, repository contexts, and execution feedback. However, the native execution environments of these harnesses are inherently misaligned with policy-gradient training: environmental crashes and reward hacking corrupt outcome signals, while train-inference discrepancies decouple rollout behavior from policy updates. To address this, we present LEGO-RL, a framework that bridges native coding-agent harnesses with scalable policy-gradient optimization without modifying their internal control flow. LEGO-RL is built upon three pillars: (1) faithful optimization via in-process LLM proxying that captures raw generation streams for token-level alignment and robust trainer-side log-probability recomputation, even under harness-side compaction or re-serialization; (2) reliable execution via scalable sandbox orchestration featuring image caching and stage-wise defenses to mitigate reward hacking; and (3) observable training through an integrated plugin that automates validation and monitoring, paired with a Live UI for granular trajectory diagnostics. We evaluate LEGO-RL by training the sparse MoE model Qwen3.5-35B-A3B with GSPO across three native coding-agent harnesses. LEGO-RL improves Qwen3.5-35B-A3B across OpenHands SDK (64.0% to 70.4%), Claude Code (62.4% to 68.2%), and OpenCode (57.2% to 66.6%) on SWE-bench Verified, while maintaining a rollout-training probability correlation above 0.99.","upvotes":2,"discussionId":"6a855dd4536bdd3bdd48f936","projectPage":"https://legox.net/","githubRepo":"https://github.com/LegoX/Lego-RL","githubRepoAddedBy":"user","ai_summary":"LEGO-RL connects native coding-agent harnesses to scalable policy-gradient training via in-process LLM proxying, sandbox orchestration, and integrated monitoring, improving sparse MoE model performance across multiple harnesses.","ai_keywords":["reinforcement learning","policy-gradient optimization","LLM proxying","log-probability recomputation","sandbox orchestration","reward hacking","sparse MoE","GSPO","SWE-bench Verified"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":5,"organization":{"_id":"6951e68428a36c372970db39","name":"Lego-X","fullname":"LegoX","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/60fc2fcca6bdebbe52dfdaf4/4TrxKnhwlnePzhe1YzGhb.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"63c20105726f62e411fbe882","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63c20105726f62e411fbe882/2UsU9O2psbDjJzz-sAmGH.jpeg","isPro":false,"fullname":"Yuxin Jiang","user":"YuxinJiang","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6951e68428a36c372970db39","name":"Lego-X","fullname":"LegoX","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/60fc2fcca6bdebbe52dfdaf4/4TrxKnhwlnePzhe1YzGhb.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.17393.md","query":{}}">
LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents
Abstract
LEGO-RL connects native coding-agent harnesses to scalable policy-gradient training via in-process LLM proxying, sandbox orchestration, and integrated monitoring, improving sparse MoE model performance across multiple harnesses.
Reinforcement learning for coding agents increasingly relies on long-running agent harnesses to manage tool integration, repository contexts, and execution feedback. However, the native execution environments of these harnesses are inherently misaligned with policy-gradient training: environmental crashes and reward hacking corrupt outcome signals, while train-inference discrepancies decouple rollout behavior from policy updates. To address this, we present LEGO-RL, a framework that bridges native coding-agent harnesses with scalable policy-gradient optimization without modifying their internal control flow. LEGO-RL is built upon three pillars: (1) faithful optimization via in-process LLM proxying that captures raw generation streams for token-level alignment and robust trainer-side log-probability recomputation, even under harness-side compaction or re-serialization; (2) reliable execution via scalable sandbox orchestration featuring image caching and stage-wise defenses to mitigate reward hacking; and (3) observable training through an integrated plugin that automates validation and monitoring, paired with a Live UI for granular trajectory diagnostics. We evaluate LEGO-RL by training the sparse MoE model Qwen3.5-35B-A3B with GSPO across three native coding-agent harnesses. LEGO-RL improves Qwen3.5-35B-A3B across OpenHands SDK (64.0% to 70.4%), Claude Code (62.4% to 68.2%), and OpenCode (57.2% to 66.6%) on SWE-bench Verified, while maintaining a rollout-training probability correlation above 0.99.
Community
We are happy to announce Lego-RL: an open-source framework for harness-native reinforcement learning of coding agents. All code, data, and models are available on GitHub and Hugging Face.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.17393 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.17393 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.17393 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.