QVal is a training-free testbed that directly evaluates dense supervision signals for LLM agents by measuring how well they order actions according to a reference policy's Q-values, revealing that simple prompting baselines consistently beat recent more complex methods from the literature, without any expensive training run.</p>\n","updatedAt":"2026-07-01T12:20:50.036Z","author":{"_id":"644555c72d91b15b4c7ebd1c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/644555c72d91b15b4c7ebd1c/28zmmIkLHUUiQXQ3RQlPM.jpeg","fullname":"Matteo Merler","name":"merlerm","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9087345004081726},"editors":["merlerm"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/644555c72d91b15b4c7ebd1c/28zmmIkLHUUiQXQ3RQlPM.jpeg"],"reactions":[],"isReport":false}},{"id":"6a45c34f5e704bf210cff6bc","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":372,"isUserFollowing":false},"createdAt":"2026-07-02T01:47:59.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [HINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agents](https://huggingface.co/papers/2605.17873) (2026)\n* [Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents](https://huggingface.co/papers/2606.26080) (2026)\n* [What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents](https://huggingface.co/papers/2605.19447) (2026)\n* [Multi-Rollout On-Policy Distillation via Peer Successes and Failures](https://huggingface.co/papers/2605.12652) (2026)\n* [PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents](https://huggingface.co/papers/2606.16215) (2026)\n* [Self-Induced Outcome Potential: Turn-Level Credit Assignment for Agents without Verifiers](https://huggingface.co/papers/2605.04984) (2026)\n* [Self-evolving LLM agents with in-distribution Optimization](https://huggingface.co/papers/2606.07367) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"<p>This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2605.17873\">HINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.26080\">Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.19447\">What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.12652\">Multi-Rollout On-Policy Distillation via Peer Successes and Failures</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.16215\">PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.04984\">Self-Induced Outcome Potential: Turn-Level Credit Assignment for Agents without Verifiers</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.07367\">Self-evolving LLM agents with in-distribution Optimization</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code>@librarian-bot recommend</code></p>\n","updatedAt":"2026-07-02T01:47:59.524Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":372,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7551752924919128},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.32034","authors":[{"_id":"6a4502ad4f1dd35e48fb8c83","user":{"_id":"65901210ccbc1e2cc7d192ca","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65901210ccbc1e2cc7d192ca/yX3QJ5keD9b6kGo1DdwCo.jpeg","isPro":false,"fullname":"Sergio Hernandez","user":"sergio-hernandez","type":"user","name":"sergio-hernandez"},"name":"Sergio Hernández-Gutiérrez","status":"claimed_verified","statusLastChangedAt":"2026-07-01T13:31:58.535Z","hidden":false},{"_id":"6a4502ad4f1dd35e48fb8c84","user":{"_id":"644555c72d91b15b4c7ebd1c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/644555c72d91b15b4c7ebd1c/28zmmIkLHUUiQXQ3RQlPM.jpeg","isPro":false,"fullname":"Matteo Merler","user":"merlerm","type":"user","name":"merlerm"},"name":"Matteo Merler","status":"claimed_verified","statusLastChangedAt":"2026-07-01T13:54:16.460Z","hidden":false},{"_id":"6a4502ad4f1dd35e48fb8c85","name":"Ilze Amanda Auzina","hidden":false},{"_id":"6a4502ad4f1dd35e48fb8c86","name":"Joschka Strüber","hidden":false},{"_id":"6a4502ad4f1dd35e48fb8c87","name":"Ameya Prabhu","hidden":false},{"_id":"6a4502ad4f1dd35e48fb8c88","name":"Matthias Bethge","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/644555c72d91b15b4c7ebd1c/00qtf4A7-tTGeVpfRBTwY.png"],"publishedAt":"2026-06-30T00:00:00.000Z","submittedOnDailyAt":"2026-07-01T00:00:00.000Z","title":"QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents","submittedOnDailyBy":{"_id":"644555c72d91b15b4c7ebd1c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/644555c72d91b15b4c7ebd1c/28zmmIkLHUUiQXQ3RQlPM.jpeg","isPro":false,"fullname":"Matteo Merler","user":"merlerm","type":"user","name":"merlerm"},"summary":"LLM agents increasingly act over long horizons, where a single trajectory can contain hundreds or thousands of actions. In these settings, outcome-only rewards provide too sparse guidance, failing to inform the model about the goodness of intermediate actions. Dense supervision methods aim to solve this problem by scoring intermediate steps, from intrinsic confidence to self-distillation and embedding similarities. However, it is common practice to evaluate them by measuring the downstream performance of a training pipeline that integrates them. This is expensive, conflates supervision quality with training engineering confounders, and renders different methodological families requiring distinct training setups incomparable. As a result, dense supervision methods are rarely benchmarked on common ground. We introduce QVal, a training-free testbed for directly evaluating dense supervision signals. Given a state-action pair, QVal measures how well a method's score is Q-aligned: whether it orders actions according to the Q-values of a strong reference-policy. This lets us compare signals before any training run and separate signal quality from other engineering choices. We instantiate QVal as QVal-v1.0, benchmarking 21 dense supervision methods across four diverse environments and seven methodological families, with over 1.2K evaluation experiments across six open-weight model backbones. We find that simple prompting baselines consistently outperform recent dense supervision methods from the literature, and that performance clusters strongly by family. These findings hold across model sizes, environments, and observation modalities. QVal is designed to be easily extensible to new environments and methods, enabling researchers to iterate on dense supervision methods before any training run.","upvotes":9,"discussionId":"6a4502ae4f1dd35e48fb8c89","projectPage":"https://q-val.com","githubRepo":"https://github.com/bethgelab/qval","githubRepoAddedBy":"user","ai_summary":"A testbed called QVal is introduced for evaluating dense supervision signals in long-horizon LLM agent tasks by measuring how well method scores align with Q-values, enabling fair comparison of different supervision approaches without training.","ai_keywords":["LLM agents","dense supervision","Q-values","training-free","Q-aligned","supervision signals","reference-policy","long horizons","action sequences","model backbones"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":3,"organization":{"_id":"65745368a4ee9a4fe778ed92","name":"bethgelab","fullname":"Bethgelab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6304da46ce6b12280b1bd575/XH-3Q4A8p-P7JFR1LxSgx.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"644555c72d91b15b4c7ebd1c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/644555c72d91b15b4c7ebd1c/28zmmIkLHUUiQXQ3RQlPM.jpeg","isPro":false,"fullname":"Matteo Merler","user":"merlerm","type":"user"},{"_id":"65901210ccbc1e2cc7d192ca","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65901210ccbc1e2cc7d192ca/yX3QJ5keD9b6kGo1DdwCo.jpeg","isPro":false,"fullname":"Sergio Hernandez","user":"sergio-hernandez","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"},{"_id":"656202582c671784e2cf1440","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4cpkcwGn3Z6oKquShOXqC.jpeg","isPro":false,"fullname":"Sebastian Dziadzio","user":"sbdzdz","type":"user"},{"_id":"60d9e5b71fa5d458da777550","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1676481484908-60d9e5b71fa5d458da777550.png","isPro":false,"fullname":"Giovanni Bonetta","user":"giobin","type":"user"},{"_id":"6464a0d41683d3c81f51924a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6464a0d41683d3c81f51924a/s7yYVwfUB4WOhVFJS6A6T.jpeg","isPro":false,"fullname":"Ameya Prabhu","user":"AmeyaPrabhu","type":"user"},{"_id":"6058e839c0e16cb4d7e25ec9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6058e839c0e16cb4d7e25ec9/xm-vhYRePdkJNmlCfetHk.jpeg","isPro":false,"fullname":"Manuel de Prada","user":"manueldeprada","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"655f577fc073ea9cf86a3298","avatarUrl":"/avatars/509a9c3c5564028907c30ce762a1c86e.svg","isPro":false,"fullname":"Pietro Ferrazzi","user":"ferrazzipietro","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"65745368a4ee9a4fe778ed92","name":"bethgelab","fullname":"Bethgelab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6304da46ce6b12280b1bd575/XH-3Q4A8p-P7JFR1LxSgx.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.32034.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2606.32034 in a model README.md to link it from this page.

Cite arxiv.org/abs/2606.32034 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.