<a href=\"https://cdn-uploads.huggingface.co/production/uploads/6602b210746f31a6443075c0/-ezi819oEeeW_77tmHEHW.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/6602b210746f31a6443075c0/-ezi819oEeeW_77tmHEHW.png\" alt=\"final_intern_副本\"></a><br>Generating professional scholarly content, such as peer reviews and rebuttals, requires an intricate synergy between domain reasoning and factual grounding. This work presents a comprehensive framework for the development and evaluation of specialized scholarly agents, InternReviewer and InternAdvocate. We first establish a large-scale, high-quality scholarly dataset and integrate a high-efficiency arXiv retrieval tool to enable active evidence gathering. To optimize these agents, we implement an agentic Reinforcement Learning (RL) paradigm driven by a unified objective metric and reward system. This system avoids the biases of subjective model-based judging by employing multi-dimensional criteria, including reference-anchored semantic alignment, structural compliance, and a strict verification mechanism that cross-checks citations against real-time interaction logs to eliminate hallucinations. Experimental results demonstrate that agents trained within this closed-loop framework exhibit significant improvements in reasoning depth and citation accuracy.</p>\n<h1 class=\"relative group flex items-baseline\">\n\t<a id=\"data-collection-and-curation\" class=\"block pr-1.5 text-lg md:absolute md:p-1.5 md:opacity-0 md:group-hover:opacity-100 md:right-full\" href=\"#data-collection-and-curation\" rel=\"nofollow\">\n\t\t<span class=\"header-link\"><svg class=\"text-gray-500 hover:text-black dark:hover:text-gray-200 w-4\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" aria-hidden=\"true\" role=\"img\" width=\"1em\" height=\"1em\" preserveAspectRatio=\"xMidYMid meet\" viewBox=\"0 0 256 256\"><path d=\"M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z\" fill=\"currentColor\"></path></svg></span>\n\t</a>\n\t<span>\n\t\tData Collection and Curation\n\t</span>\n</h1>\n<p>Leveraging the official OpenReview API (<a href=\"https://github.com/openreview/openreview-py\" rel=\"nofollow\">https://github.com/openreview/openreview-py</a>), we systematically curate a large-scale collection of peer review and rebuttal trajectories for manuscripts published through 2025. Our collection encompasses flagship computer science conferences, specifically ICLR (2013–2025), NeurIPS (2021–2024), and ICML (2025), resulting in a total of 184,857 unique review-rebuttal pairs alongside their original manuscript PDFs. See figure below for the relevant review and rebuttal data collection and curation pipeline.<br><a href=\"https://cdn-uploads.huggingface.co/production/uploads/6602b210746f31a6443075c0/eFoUK0SLdh__9lbGb91DV.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/6602b210746f31a6443075c0/eFoUK0SLdh__9lbGb91DV.png\" alt=\"data_curation_2\"></a></p>\n<h1 class=\"relative group flex items-baseline\">\n\t<a id=\"agentic-reinforcement-learning-framework\" class=\"block pr-1.5 text-lg md:absolute md:p-1.5 md:opacity-0 md:group-hover:opacity-100 md:right-full\" href=\"#agentic-reinforcement-learning-framework\" rel=\"nofollow\">\n\t\t<span class=\"header-link\"><svg class=\"text-gray-500 hover:text-black dark:hover:text-gray-200 w-4\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" aria-hidden=\"true\" role=\"img\" width=\"1em\" height=\"1em\" preserveAspectRatio=\"xMidYMid meet\" viewBox=\"0 0 256 256\"><path d=\"M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z\" fill=\"currentColor\"></path></svg></span>\n\t</a>\n\t<span>\n\t\tAgentic Reinforcement Learning Framework\n\t</span>\n</h1>\n<p>The following figure presents the Agentic Reinforcement Learning framework behind InternReviewer and InternAdvocate. The framework enables agents to interact with a time-aware scholarly retrieval environment, perform evidence-grounded reasoning, and optimize their behaviors through a decomposed objective reward covering semantic quality, structural compliance, tool usage, and citation verification. This closed-loop RL paradigm trains specialized academic agents capable of generating more rigorous, grounded, and reliable peer reviews and rebuttals.<br><a href=\"https://cdn-uploads.huggingface.co/production/uploads/6602b210746f31a6443075c0/9ySVTVovDir74x9eh78T3.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/6602b210746f31a6443075c0/9ySVTVovDir74x9eh78T3.png\" alt=\"RL_framework_0421\"></a></p>\n","updatedAt":"2026-09-02T01:33:56.260Z","author":{"_id":"6602b210746f31a6443075c0","avatarUrl":"/avatars/07bc684e8981d3827976ecf13f89e11d.svg","fullname":"XueruiSu","name":"Xuerui2312","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8359649181365967},"editors":["Xuerui2312"],"editorAvatarUrls":["/avatars/07bc684e8981d3827976ecf13f89e11d.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.28612","authors":[{"_id":"6a969f40cd6ebc484732efef","user":{"_id":"6602b210746f31a6443075c0","avatarUrl":"/avatars/07bc684e8981d3827976ecf13f89e11d.svg","isPro":false,"fullname":"XueruiSu","user":"Xuerui2312","type":"user","name":"Xuerui2312"},"name":"Xuerui Su","status":"claimed_verified","statusLastChangedAt":"2026-09-01T16:45:04.882Z","hidden":false},{"_id":"6a969f40cd6ebc484732eff0","name":"Liya Guo","hidden":false},{"_id":"6a969f40cd6ebc484732eff1","name":"Qizhi Pei","hidden":false},{"_id":"6a969f40cd6ebc484732eff2","name":"Qipeng Guo","hidden":false},{"_id":"6a969f40cd6ebc484732eff3","name":"Zhongbo Tian","hidden":false},{"_id":"6a969f40cd6ebc484732eff4","name":"Lijun Wu","hidden":false},{"_id":"6a969f40cd6ebc484732eff5","name":"Kai Chen","hidden":false},{"_id":"6a969f40cd6ebc484732eff6","name":"Zun Wang","hidden":false}],"publishedAt":"2026-07-21T00:00:00.000Z","submittedOnDailyAt":"2026-09-02T00:00:00.000Z","title":"InternReviewer & InternAdvocate: Objective Reward and Evaluation for Agentic Reinforcement Learning in Peer Review and Rebuttal","submittedOnDailyBy":{"_id":"6602b210746f31a6443075c0","avatarUrl":"/avatars/07bc684e8981d3827976ecf13f89e11d.svg","isPro":false,"fullname":"XueruiSu","user":"Xuerui2312","type":"user","name":"Xuerui2312"},"summary":"Generating professional scholarly content, such as peer reviews and rebuttals, requires an intricate synergy between domain reasoning and factual grounding. This work presents a comprehensive framework for the development and evaluation of specialized scholarly agents, InternReviewer and InternAdvocate. We first establish a large-scale, high-quality scholarly dataset and integrate a high-efficiency arXiv retrieval tool to enable active evidence gathering. To optimize these agents, we implement an agentic Reinforcement Learning (RL) paradigm driven by a unified objective metric and reward system. This system avoids the biases of subjective model-based judging by employing multi-dimensional criteria, including reference-anchored semantic alignment, structural compliance, and a strict verification mechanism that cross-checks citations against real-time interaction logs to eliminate hallucinations. Experimental results demonstrate that agents trained within this closed-loop framework exhibit significant improvements in reasoning depth and citation accuracy.","upvotes":5,"discussionId":"6a969f40cd6ebc484732eff7","ai_summary":"Specialized scholarly agents use reinforcement learning and real-time citation verification to improve reasoning and factual accuracy in peer review and rebuttal generation.","ai_keywords":["agentic Reinforcement Learning","reward system","reference-anchored semantic alignment","structural compliance","citation verification","hallucination elimination","closed-loop framework"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"6a4fb75a1c66dbf208e7ddb6","name":"Shanghai-AI-Laboratory","fullname":"Shanghai AI Laboratory","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65cd955637be1841d0b75397/Rao_Kq6NMtTVfSqLUIR4k.webp"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6602b210746f31a6443075c0","avatarUrl":"/avatars/07bc684e8981d3827976ecf13f89e11d.svg","isPro":false,"fullname":"XueruiSu","user":"Xuerui2312","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"},{"_id":"641ea59e2c631e05c2cdafc7","avatarUrl":"/avatars/aed1e65c2ef652d193ce8fb7623f3b93.svg","isPro":false,"fullname":"wz22","user":"wz22","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"6397f6081323f19c578f142e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6397f6081323f19c578f142e/it7FYYKjlLX8wSsMLm8EO.jpeg","isPro":false,"fullname":"QizhiPei","user":"QizhiPei","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6a4fb75a1c66dbf208e7ddb6","name":"Shanghai-AI-Laboratory","fullname":"Shanghai AI Laboratory","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65cd955637be1841d0b75397/Rao_Kq6NMtTVfSqLUIR4k.webp"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.28612.md","query":{}}">
InternReviewer & InternAdvocate: Objective Reward and Evaluation for Agentic Reinforcement Learning in Peer Review and Rebuttal
Abstract
Specialized scholarly agents use reinforcement learning and real-time citation verification to improve reasoning and factual accuracy in peer review and rebuttal generation.
Generating professional scholarly content, such as peer reviews and rebuttals, requires an intricate synergy between domain reasoning and factual grounding. This work presents a comprehensive framework for the development and evaluation of specialized scholarly agents, InternReviewer and InternAdvocate. We first establish a large-scale, high-quality scholarly dataset and integrate a high-efficiency arXiv retrieval tool to enable active evidence gathering. To optimize these agents, we implement an agentic Reinforcement Learning (RL) paradigm driven by a unified objective metric and reward system. This system avoids the biases of subjective model-based judging by employing multi-dimensional criteria, including reference-anchored semantic alignment, structural compliance, and a strict verification mechanism that cross-checks citations against real-time interaction logs to eliminate hallucinations. Experimental results demonstrate that agents trained within this closed-loop framework exhibit significant improvements in reasoning depth and citation accuracy.
Community
Generating professional scholarly content, such as peer reviews and rebuttals, requires an intricate synergy between domain reasoning and factual grounding. This work presents a comprehensive framework for the development and evaluation of specialized scholarly agents, InternReviewer and InternAdvocate. We first establish a large-scale, high-quality scholarly dataset and integrate a high-efficiency arXiv retrieval tool to enable active evidence gathering. To optimize these agents, we implement an agentic Reinforcement Learning (RL) paradigm driven by a unified objective metric and reward system. This system avoids the biases of subjective model-based judging by employing multi-dimensional criteria, including reference-anchored semantic alignment, structural compliance, and a strict verification mechanism that cross-checks citations against real-time interaction logs to eliminate hallucinations. Experimental results demonstrate that agents trained within this closed-loop framework exhibit significant improvements in reasoning depth and citation accuracy.
Data Collection and Curation
Leveraging the official OpenReview API (https://github.com/openreview/openreview-py), we systematically curate a large-scale collection of peer review and rebuttal trajectories for manuscripts published through 2025. Our collection encompasses flagship computer science conferences, specifically ICLR (2013–2025), NeurIPS (2021–2024), and ICML (2025), resulting in a total of 184,857 unique review-rebuttal pairs alongside their original manuscript PDFs. See figure below for the relevant review and rebuttal data collection and curation pipeline.
Agentic Reinforcement Learning Framework
The following figure presents the Agentic Reinforcement Learning framework behind InternReviewer and InternAdvocate. The framework enables agents to interact with a time-aware scholarly retrieval environment, perform evidence-grounded reasoning, and optimize their behaviors through a decomposed objective reward covering semantic quality, structural compliance, tool usage, and citation verification. This closed-loop RL paradigm trains specialized academic agents capable of generating more rigorous, grounded, and reliable peer reviews and rebuttals.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.28612 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.28612 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.28612 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.