On ITBench, Qwen3.5-27B with an evolved harness beats GPT-5.5 on the baseline harness by 19.2 points.</p>\n<p>More capable models do not always make better enterprise agents.</p>\n<p>With the right harness, a smaller open-weight model can outperform a larger frontier model running the default scaffold.</p>\n<p>In our new preprint, we present StarHarness, a method for directly searching for an environment-optimal harness around a fixed model. It evolves prompts, tools, schemas, execution logic, state handling, skills, MCPs, subagents, context management, and more, while keeping model weights fixed.</p>\n<p>Across three enterprise benchmarks, the evolved harness improved performance by 20–35 percentage points and reduced inference cost by 17–53%.</p>\n<p>Our results suggest that some of the capability gap we attribute to models may actually come from how well the surrounding harness fits the environment.</p>\n","updatedAt":"2026-08-31T05:53:20.614Z","author":{"_id":"6333213e66b27c4ad195bbc7","avatarUrl":"/avatars/0f911fee8c173543a6e0cbd78a940676.svg","fullname":"Esakkivel Esakkiraja","name":"dotvignesh","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.908681333065033},"editors":["dotvignesh"],"editorAvatarUrls":["/avatars/0f911fee8c173543a6e0cbd78a940676.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.24804","authors":[{"_id":"6a950eb5073195fee51572ab","name":"Esakkivel Esakkiraja","hidden":false},{"_id":"6a950eb5073195fee51572ac","name":"Denis Akhiyarov","hidden":false},{"_id":"6a950eb5073195fee51572ad","name":"Vikas Yadav","hidden":false},{"_id":"6a950eb5073195fee51572ae","name":"Sai Rajeswar","hidden":false},{"_id":"6a950eb5073195fee51572af","name":"Patrice Bechard","hidden":false},{"_id":"6a950eb5073195fee51572b0","name":"Sridhar Nemala","hidden":false},{"_id":"6a950eb5073195fee51572b1","name":"Sagar Davasam","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/6333213e66b27c4ad195bbc7/RWpgMiqbMy3Do-VTjI6Dq.png"],"publishedAt":"2026-08-25T00:00:00.000Z","submittedOnDailyAt":"2026-08-31T00:00:00.000Z","title":"StarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments","submittedOnDailyBy":{"_id":"6333213e66b27c4ad195bbc7","avatarUrl":"/avatars/0f911fee8c173543a6e0cbd78a940676.svg","isPro":false,"fullname":"Esakkivel Esakkiraja","user":"dotvignesh","type":"user","name":"dotvignesh"},"summary":"We present StarHarness, a framework for evolving environment-specific agent harnesses while keeping model weights fixed. The evolved harness can include prompt and task framing, tool interfaces, skills, MCP-backed providers, subagent structure, and agent-loop configuration. StarHarness constructs a compact evolution pool by stratifying tasks according to baseline failure behavior, separates proposer-visible search tasks from proposer-hidden selection tasks, and reserves held-out tasks for evaluating generalization. Across ITBench SRE, EnterpriseOps-Gym ITSM, and AutomationBench Finance, harness evolution improves full-benchmark performance by 20-35 percentage points over the default harness after 4-12 accepted changes per environment. These gains persist on tasks excluded from evolution and transfer without re-evolution across GPT and Qwen model families. Trace analysis links the improvements to interface repairs, environment conventions, and operational knowledge that compresses search, with fewer false-positive diagnoses and shorter trajectories in several settings. StarHarness therefore offers a practical way to reduce persistent model-environment mismatch in tool-rich enterprise tasks.","upvotes":3,"discussionId":"6a950eb5073195fee51572b2","ai_summary":"StarHarness evolves fixed-weight agent harnesses via stratified task pools and hidden selection to improve enterprise tool-use performance and cross-model transfer.","ai_keywords":["agent harnesses","MCP-backed providers","subagent structure","agent-loop configuration","evolution pool","stratifying tasks","baseline failure behavior","proposer-visible search tasks","proposer-hidden selection tasks","held-out tasks","ITBench SRE","EnterpriseOps-Gym ITSM","AutomationBench Finance","harness evolution","GPT","Qwen","false-positive diagnoses","model-environment mismatch"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"65f4df5de83b55da5d79fbb6","name":"ServiceNow-AI","fullname":"ServiceNow-AI","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/63d3095c2727d7888cbb54e2/Uv-Lx8PVGviqokfOyYlCN.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6333213e66b27c4ad195bbc7","avatarUrl":"/avatars/0f911fee8c173543a6e0cbd78a940676.svg","isPro":false,"fullname":"Esakkivel Esakkiraja","user":"dotvignesh","type":"user"},{"_id":"65c20ee58aedd6edd2b89000","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65c20ee58aedd6edd2b89000/LtS4YTbmxiCFqHSGHfdC8.png","isPro":false,"fullname":"Chmielewski","user":"Eryk-Chmielewski","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"65f4df5de83b55da5d79fbb6","name":"ServiceNow-AI","fullname":"ServiceNow-AI","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/63d3095c2727d7888cbb54e2/Uv-Lx8PVGviqokfOyYlCN.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.24804.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2608.24804 in a model README.md to link it from this page.

Cite arxiv.org/abs/2608.24804 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2608.24804 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.