E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation<br>Presented as one of the demos in the Qwen3.8-Max Blog: <a href=\"https://qwen.ai/blog?id=qwen3.8\" rel=\"nofollow\">https://qwen.ai/blog?id=qwen3.8</a></p>\n","updatedAt":"2026-09-02T02:22:38.828Z","author":{"_id":"643e9018e1b2a57ff0d50e65","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/643e9018e1b2a57ff0d50e65/T-dgJGgGGdlYyS18DGuce.jpeg","fullname":"Xinjie Shen","name":"Frinkleko","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":7,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.7715960144996643},"editors":["Frinkleko"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/643e9018e1b2a57ff0d50e65/T-dgJGgGGdlYyS18DGuce.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.30730","authors":[{"_id":"6a9787e9fe3c2f89286c393d","name":"Wei Fan","hidden":false},{"_id":"6a9787e9fe3c2f89286c393e","name":"Xinjie Shen","hidden":false},{"_id":"6a9787e9fe3c2f89286c393f","name":"Xudong Guo","hidden":false},{"_id":"6a9787e9fe3c2f89286c3940","name":"Jianhong Tu","hidden":false},{"_id":"6a9787e9fe3c2f89286c3941","name":"Yang Su","hidden":false},{"_id":"6a9787e9fe3c2f89286c3942","name":"Yinger Zhang","hidden":false},{"_id":"6a9787e9fe3c2f89286c3943","name":"Lianghao Deng","hidden":false},{"_id":"6a9787e9fe3c2f89286c3944","name":"Fengyu Wang","hidden":false},{"_id":"6a9787e9fe3c2f89286c3945","name":"Baohua Dong","hidden":false},{"_id":"6a9787e9fe3c2f89286c3946","name":"Yangqiu Song","hidden":false},{"_id":"6a9787e9fe3c2f89286c3947","name":"Dayiheng Liu","hidden":false}],"publishedAt":"2026-08-31T00:00:00.000Z","submittedOnDailyAt":"2026-09-02T00:00:00.000Z","title":"E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation","submittedOnDailyBy":{"_id":"643e9018e1b2a57ff0d50e65","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/643e9018e1b2a57ff0d50e65/T-dgJGgGGdlYyS18DGuce.jpeg","isPro":false,"fullname":"Xinjie Shen","user":"Frinkleko","type":"user","name":"Frinkleko"},"summary":"Long-horizon agentic tasks go beyond chaining short tasks over more interaction turns. Their evolving dynamic environments and long-range dependencies require Large Language Models (LLMs) to continually explore, learn from experience, and adapt their policies over thousands of steps. We introduce E-Commerce Bench, the first open-source benchmark that integrates multi-round counterpart negotiation and dynamic events into a year-long business operation. Over a 365-day year, an LLM agent concurrently runs multiple online stores, researching the market, negotiating with suppliers to source inventory, optimizing sales strategies, fulfilling orders, handling returns, and managing cash flow to maximize its end-of-year total assets. To construct a realistic merchant-side operating environment, the product and supplier data are derived from a real e-commerce platform, while a year-long calendar of promotions, natural disasters, and supply-chain shocks continually reshapes demand. For reproducibility, both sides of the market are deterministic: customer purchases and returns follow a fixed demand model, while a negotiation kernel determines supplier pricing, concessions, and decisions, with an LLM used only to verbalize them. We evaluate 18 frontier models across seven dimensions, including year-end assets, and find that no single model dominates. GPT-5.6 Sol earns the most, growing the 100,000 opening stake into 1,431,425, yet it ranks 16th of 18 on fraud avoidance and trails Fable5 in operational efficiency. Among open-weight models, Qwen3.8-Max-Preview leads with 416,252, 38% above GLM 5.2 (high), and achieves the strongest learning over the horizon, progressively bargaining down prices across repeated orders. Our code is available at https://github.com/QwenLM/E-CommerceBench.","upvotes":4,"discussionId":"6a9787e9fe3c2f89286c3948","projectPage":"https://ecbench.github.io/","githubRepo":"https://github.com/QwenLM/E-CommerceBench","githubRepoAddedBy":"user","ai_summary":"A year-long e-commerce benchmark evaluates LLM agents on multi-store negotiation, dynamic market events, and long-horizon policy adaptation across 18 frontier models.","ai_keywords":["Large Language Models","LLM agent","negotiation kernel","demand model","open-weight models","multi-round negotiation","dynamic events"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":5,"organization":{"_id":"64c8b5837fe12ecd0a7e92eb","name":"Qwen","fullname":"Qwen","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6215ca5692c0ecfba9186921/hrRM50-6XcdWgg2AKpENG.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"643e9018e1b2a57ff0d50e65","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/643e9018e1b2a57ff0d50e65/T-dgJGgGGdlYyS18DGuce.jpeg","isPro":false,"fullname":"Xinjie Shen","user":"Frinkleko","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"653882396d6743c014db1fe5","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/saCRVks8jnS2Ifs3mT7QW.png","isPro":false,"fullname":"FAN WEI","user":"AlexFanWei","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"64c8b5837fe12ecd0a7e92eb","name":"Qwen","fullname":"Qwen","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6215ca5692c0ecfba9186921/hrRM50-6XcdWgg2AKpENG.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.30730.md","query":{}}">
E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation
Abstract
A year-long e-commerce benchmark evaluates LLM agents on multi-store negotiation, dynamic market events, and long-horizon policy adaptation across 18 frontier models.
Long-horizon agentic tasks go beyond chaining short tasks over more interaction turns. Their evolving dynamic environments and long-range dependencies require Large Language Models (LLMs) to continually explore, learn from experience, and adapt their policies over thousands of steps. We introduce E-Commerce Bench, the first open-source benchmark that integrates multi-round counterpart negotiation and dynamic events into a year-long business operation. Over a 365-day year, an LLM agent concurrently runs multiple online stores, researching the market, negotiating with suppliers to source inventory, optimizing sales strategies, fulfilling orders, handling returns, and managing cash flow to maximize its end-of-year total assets. To construct a realistic merchant-side operating environment, the product and supplier data are derived from a real e-commerce platform, while a year-long calendar of promotions, natural disasters, and supply-chain shocks continually reshapes demand. For reproducibility, both sides of the market are deterministic: customer purchases and returns follow a fixed demand model, while a negotiation kernel determines supplier pricing, concessions, and decisions, with an LLM used only to verbalize them. We evaluate 18 frontier models across seven dimensions, including year-end assets, and find that no single model dominates. GPT-5.6 Sol earns the most, growing the 100,000 opening stake into 1,431,425, yet it ranks 16th of 18 on fraud avoidance and trails Fable5 in operational efficiency. Among open-weight models, Qwen3.8-Max-Preview leads with 416,252, 38% above GLM 5.2 (high), and achieves the strongest learning over the horizon, progressively bargaining down prices across repeated orders. Our code is available at https://github.com/QwenLM/E-CommerceBench.
Community
E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation
Presented as one of the demos in the Qwen3.8-Max Blog: https://qwen.ai/blog?id=qwen3.8
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.30730 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.30730 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.30730 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.