Modern agents do not operate as standalone LLMs. They run inside agent harnesses that<br>manage tools, context, and control flow, which makes the harness a critical component. Our original<br>Agent Lightning work introduced a disaggregated architecture that connects arbitrary agents to<br>reinforcement learning (RL) training through an LLM endpoint proxy. Recent frameworks such as<br>verl Uni-Agent, AReaL 2.0, slime v0.3.0, and Polar have followed this proxy-based approach. Such a<br>proxy-based training approach enables RL training with the harness. In this work, we use the term<br>harnessed agentic RL to describe this paradigm, in which the deploy-time harness is directly involved<br>in model post-training, thereby narrowing the gap between training and actual use.</p>\n<p>We find that harnessed agentic RL differs fundamentally from traditional agentic RL and introduces<br>a new set of challenges. In traditional agentic RL, the training engine owns the environment<br>interaction loop. In harnessed agentic RL, the harness owns this loop, while the training engine<br>observes only a sequence of LLM request-response pairs. How to model and assemble these calls into<br>training samples remains an open question. Through a careful study, we identify several challenges of<br>harnessed agentic RL, including retokenization, sample merging, advantage calculation, loss<br>normalization, and training backend scheduling. We find that, if not properly addressed, these<br>challenges can lead to ineffective or unstable training. Existing frameworks generally leave these<br>issues underspecified. In this paper, we provide the first comprehensive elaboration of them.</p>\n<p>We further present Agent Lightning v1.0, a lightweight framework for harnessed agentic RL. We<br>treat simplicity as a first principle, implementing the framework in only approximately 3,500 lines of<br>code. Its compact design supports arbitrary agent harnesses and provides a practical testbed for<br>studying these challenges. We validate Agent Lightning v1.0 on general instruction-following agent,<br>search agent, and coding agent. For coding agent, we find that existing RL frameworks provide<br>limited support, including a lack of data and complete training scripts, as well as a reliance on<br>large-scale computational resources. To address this gap, we provide a complete data-cleaning<br>pipeline and reproducible training scripts based on open-source dataset and models. Using only 6K<br>training examples and modest compute, RL improves Qwen3.5-9B on SWE-bench Verified from<br>41.8% to 56.4%, an absolute 14.6% gain. We release the complete workflow and scripts to facilitate<br>reproducible harnessed agentic RL in Agent Lightning v1.0.</p>\n","updatedAt":"2026-08-19T04:27:19.240Z","author":{"_id":"6455f5cababbbbd3486d6ee3","avatarUrl":"/avatars/b6c8f65fd2bef8a00aa3269856ea238e.svg","fullname":"Zhiyuan He","name":"hzy46","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9187301993370056},"editors":["hzy46"],"editorAvatarUrls":["/avatars/b6c8f65fd2bef8a00aa3269856ea238e.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.17528","authors":[{"_id":"6a853023536bdd3bdd48f86e","name":"Zhiyuan He","hidden":false},{"_id":"6a853023536bdd3bdd48f86f","name":"Siwei Zhang","hidden":false},{"_id":"6a853023536bdd3bdd48f870","name":"Zhiwen Zhou","hidden":false},{"_id":"6a853023536bdd3bdd48f871","name":"Yuqing Yang","hidden":false},{"_id":"6a853023536bdd3bdd48f872","name":"Yu Kang","hidden":false},{"_id":"6a853023536bdd3bdd48f873","name":"Yuge Zhang","hidden":false},{"_id":"6a853023536bdd3bdd48f874","name":"Luna K. Qiu","hidden":false},{"_id":"6a853023536bdd3bdd48f875","name":"Tin Yan Tsui","hidden":false},{"_id":"6a853023536bdd3bdd48f876","name":"Jiahang Xu","hidden":false},{"_id":"6a853023536bdd3bdd48f877","name":"Chong Luo","hidden":false}],"publishedAt":"2026-08-18T00:00:00.000Z","submittedOnDailyAt":"2026-08-19T00:00:00.000Z","title":"Agent Lightning v1.0: Towards Harnessed Agentic RL","submittedOnDailyBy":{"_id":"6455f5cababbbbd3486d6ee3","avatarUrl":"/avatars/b6c8f65fd2bef8a00aa3269856ea238e.svg","isPro":false,"fullname":"Zhiyuan He","user":"hzy46","type":"user","name":"hzy46"},"summary":"Modern agents operate inside agent harnesses that manage tools, context, and control flow, making the harness a critical part of the agent system. Our original Agent Lightning introduced a disaggregated architecture that connects arbitrary agents to RL training through an LLM endpoint proxy, an approach later adopted by frameworks such as verl Uni-Agent, AReaL 2.0, slime, and Polar. We refer to this paradigm as harnessed agentic RL, where the deploy-time harness directly participates in model post-training. Harnessed agentic RL differs fundamentally from traditional agentic RL: the harness, rather than the training engine, owns the environment interaction loop, while the trainer observes only sequences of LLM request-response pairs. This introduces challenges in retokenization, sample merging, advantage calculation, loss normalization, and backend scheduling, which can substantially affect training stability and effectiveness. We present Agent Lightning v1.0, a lightweight framework for harnessed agentic RL implemented in approximately 3,500 lines of code. It supports arbitrary agent harnesses and serves as a practical testbed for studying these challenges. We evaluate it on instruction-following, search, and coding agents, and provide a complete reproducible pipeline for coding-agent RL. Using only 6K training examples and modest compute, RL improves Qwen3.5-9B on SWE-bench Verified from 41.8% to 56.4%, a 14.6-point absolute gain. We release the complete workflow and training scripts to facilitate reproducible research on harnessed agentic RL.","upvotes":12,"discussionId":"6a853023536bdd3bdd48f878","projectPage":"https://microsoft.github.io/agent-lightning/stable/","ai_summary":"Agent Lightning v1.0 enables reproducible reinforcement learning for arbitrary agent harnesses, substantially improving coding-agent performance with minimal data and compute.","ai_keywords":["agent harnesses","harnessed agentic RL","LLM endpoint proxy","retokenization","advantage calculation","loss normalization","Agent Lightning v1.0","SWE-bench Verified"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"5e6485f787403103f9f1055e","name":"microsoft","fullname":"Microsoft","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1583646260758-5e64858c87403103f9f1055d.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6455f5cababbbbd3486d6ee3","avatarUrl":"/avatars/b6c8f65fd2bef8a00aa3269856ea238e.svg","isPro":false,"fullname":"Zhiyuan He","user":"hzy46","type":"user"},{"_id":"62abdf657b037eafffc48808","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1655430982462-noauth.jpeg","isPro":false,"fullname":"Jiahang Xu","user":"Jiahang","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"},{"_id":"67b5ffbe60c2157ce1c51db2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67b5ffbe60c2157ce1c51db2/zpBfV1cBaMNSXuHP53MRe.jpeg","isPro":false,"fullname":"zhonglongbao","user":"zhonglongbao","type":"user"},{"_id":"690c6a5feaf5c0da350f4be2","avatarUrl":"/avatars/39d1f0d9776f8afb082802e5d3c9a3ca.svg","isPro":false,"fullname":"Siwei Zhang","user":"SiweiPro","type":"user"},{"_id":"69a4f687df4d32a46db03726","avatarUrl":"/avatars/4e1653459f8cb4e643bb1dae79624ef1.svg","isPro":false,"fullname":"yifeng wang","user":"iris98","type":"user"},{"_id":"6466d323ac657f60661d2778","avatarUrl":"/avatars/62f70630cdf1c252b80b4d5eaa5a4150.svg","isPro":false,"fullname":"Yuge Zhang","user":"ultmaster","type":"user"},{"_id":"65c4eb7cd1dcbd30d86febec","avatarUrl":"/avatars/001c8f02e8ce794b2c21883628b2da72.svg","isPro":false,"fullname":"free-bit","user":"free-bit","type":"user"},{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"6777886b8c91ac2d4b705eaf","avatarUrl":"/avatars/ac27c6728d40edd724ce99cb8369808d.svg","isPro":false,"fullname":"Wanli Li","user":"wanlilll","type":"user"},{"_id":"652ce0d4c543a08aa92e010f","avatarUrl":"/avatars/7978304e3fe99b0d4d0712441c6a24f3.svg","isPro":false,"fullname":"Haoyu Guo","user":"ghy0324","type":"user"},{"_id":"68a5f1ecc5caec9e1b51eb99","avatarUrl":"/avatars/fc2be2bdbf1c1047f54fe477378b150a.svg","isPro":false,"fullname":"Sungbin Han","user":"SungbiinHan","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"5e6485f787403103f9f1055e","name":"microsoft","fullname":"Microsoft","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1583646260758-5e64858c87403103f9f1055d.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.17528.md","query":{}}">
Agent Lightning v1.0: Towards Harnessed Agentic RL
Abstract
Agent Lightning v1.0 enables reproducible reinforcement learning for arbitrary agent harnesses, substantially improving coding-agent performance with minimal data and compute.
Modern agents operate inside agent harnesses that manage tools, context, and control flow, making the harness a critical part of the agent system. Our original Agent Lightning introduced a disaggregated architecture that connects arbitrary agents to RL training through an LLM endpoint proxy, an approach later adopted by frameworks such as verl Uni-Agent, AReaL 2.0, slime, and Polar. We refer to this paradigm as harnessed agentic RL, where the deploy-time harness directly participates in model post-training. Harnessed agentic RL differs fundamentally from traditional agentic RL: the harness, rather than the training engine, owns the environment interaction loop, while the trainer observes only sequences of LLM request-response pairs. This introduces challenges in retokenization, sample merging, advantage calculation, loss normalization, and backend scheduling, which can substantially affect training stability and effectiveness. We present Agent Lightning v1.0, a lightweight framework for harnessed agentic RL implemented in approximately 3,500 lines of code. It supports arbitrary agent harnesses and serves as a practical testbed for studying these challenges. We evaluate it on instruction-following, search, and coding agents, and provide a complete reproducible pipeline for coding-agent RL. Using only 6K training examples and modest compute, RL improves Qwen3.5-9B on SWE-bench Verified from 41.8% to 56.4%, a 14.6-point absolute gain. We release the complete workflow and training scripts to facilitate reproducible research on harnessed agentic RL.
Community
Modern agents do not operate as standalone LLMs. They run inside agent harnesses that
manage tools, context, and control flow, which makes the harness a critical component. Our original
Agent Lightning work introduced a disaggregated architecture that connects arbitrary agents to
reinforcement learning (RL) training through an LLM endpoint proxy. Recent frameworks such as
verl Uni-Agent, AReaL 2.0, slime v0.3.0, and Polar have followed this proxy-based approach. Such a
proxy-based training approach enables RL training with the harness. In this work, we use the term
harnessed agentic RL to describe this paradigm, in which the deploy-time harness is directly involved
in model post-training, thereby narrowing the gap between training and actual use.
We find that harnessed agentic RL differs fundamentally from traditional agentic RL and introduces
a new set of challenges. In traditional agentic RL, the training engine owns the environment
interaction loop. In harnessed agentic RL, the harness owns this loop, while the training engine
observes only a sequence of LLM request-response pairs. How to model and assemble these calls into
training samples remains an open question. Through a careful study, we identify several challenges of
harnessed agentic RL, including retokenization, sample merging, advantage calculation, loss
normalization, and training backend scheduling. We find that, if not properly addressed, these
challenges can lead to ineffective or unstable training. Existing frameworks generally leave these
issues underspecified. In this paper, we provide the first comprehensive elaboration of them.
We further present Agent Lightning v1.0, a lightweight framework for harnessed agentic RL. We
treat simplicity as a first principle, implementing the framework in only approximately 3,500 lines of
code. Its compact design supports arbitrary agent harnesses and provides a practical testbed for
studying these challenges. We validate Agent Lightning v1.0 on general instruction-following agent,
search agent, and coding agent. For coding agent, we find that existing RL frameworks provide
limited support, including a lack of data and complete training scripts, as well as a reliance on
large-scale computational resources. To address this gap, we provide a complete data-cleaning
pipeline and reproducible training scripts based on open-source dataset and models. Using only 6K
training examples and modest compute, RL improves Qwen3.5-9B on SWE-bench Verified from
41.8% to 56.4%, an absolute 14.6% gain. We release the complete workflow and scripts to facilitate
reproducible harnessed agentic RL in Agent Lightning v1.0.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.17528 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.17528 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.17528 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.