Can generative UI assistants keep one executable interface correct as user requirements evolve across multiple turns?</p>\n<p><strong>EvoGenUI-Bench</strong> introduces 150 five-turn tasks (750 turns) spanning information presentation, executable interaction, and tool-grounded external state. Generated artifacts are executed in a browser and evaluated using screenshots, source/DOM evidence, actor traces, and runtime logs.</p>\n<p>Across eight models, the strongest reaches 74.9% Turn Pass but completes only 37.3% of full five-turn episodes. We also introduce Adjacent Pass Retention (APR) to measure whether behavior that worked at one turn survives the next update. The results show that cross-turn maintenance—not just one-shot visual quality—is a central bottleneck for reliable generative UI.</p>\n","updatedAt":"2026-09-01T10:17:53.596Z","author":{"_id":"67e9101753eb96cb895854b1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67e9101753eb96cb895854b1/ahsrqMQYWf5bSMnDEYGn2.png","fullname":"Yue Peng","name":"pengyue-polaron","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8807159066200256},"editors":["pengyue-polaron"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/67e9101753eb96cb895854b1/ahsrqMQYWf5bSMnDEYGn2.png"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.29387","authors":[{"_id":"6a967344cd6ebc484732ee92","user":{"_id":"67e9101753eb96cb895854b1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67e9101753eb96cb895854b1/ahsrqMQYWf5bSMnDEYGn2.png","isPro":false,"fullname":"Yue Peng","user":"pengyue-polaron","type":"user","name":"pengyue-polaron"},"name":"Yue Peng","status":"claimed_verified","statusLastChangedAt":"2026-09-01T08:08:16.725Z","hidden":false},{"_id":"6a967344cd6ebc484732ee93","name":"Lanke Xia","hidden":false},{"_id":"6a967344cd6ebc484732ee94","name":"Zihan Wang","hidden":false},{"_id":"6a967344cd6ebc484732ee95","name":"Jiahao Ye","hidden":false},{"_id":"6a967344cd6ebc484732ee96","name":"Ke Ning","hidden":false},{"_id":"6a967344cd6ebc484732ee97","name":"Hongyi Wen","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/67e9101753eb96cb895854b1/7tEB8DkQgFUA6Kh8AnujJ.png"],"publishedAt":"2026-08-29T00:00:00.000Z","submittedOnDailyAt":"2026-09-01T00:00:00.000Z","title":"EvoGenUI-Bench: Evaluating LLMs as Multi-Turn Generative UI Assistants","submittedOnDailyBy":{"_id":"67e9101753eb96cb895854b1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67e9101753eb96cb895854b1/ahsrqMQYWf5bSMnDEYGn2.png","isPro":false,"fullname":"Yue Peng","user":"pengyue-polaron","type":"user","name":"pengyue-polaron"},"summary":"Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests evolve. We introduce EvoGenUI-Bench, a benchmark for multi-turn interface maintenance comprising 150 five-turn tasks and 750 turns across three scenarios: information presentation, executable interaction, and tool-grounded external state. We execute generated artifacts in a browser and evaluate them using screenshots, source and DOM evidence, actor traces, and runtime logs. Beyond turn-level and episode-level success, we measure cross-turn retention with Adjacent Pass Retention. Across eight models, even the strongest achieves 74.9% Turn Pass while completing only 37.3% of five-turn episodes; APR further falls to 52.4% on tool-grounded tasks. Diagnostic analysis shows that presentation failures center on information architecture, interaction failures on derived-state propagation and affordance binding, and tool-grounded failures additionally involve external-state grounding and requirement decomposition. These results reframe generative UI evaluation from judging isolated outputs to testing whether interface behavior, derived state, external state, and assistant claims remain synchronized as the artifact evolves.","upvotes":0,"discussionId":"6a967345cd6ebc484732ee98","githubRepo":"https://github.com/MAPS-research/EvoGenUI-Bench","githubRepoAddedBy":"user","githubStars":0,"organization":{"_id":"64467988902606605668dc8a","name":"MAPS-research","fullname":"MAPS Research","avatar":"https://www.gravatar.com/avatar/2df2a601ac42e11d30f6e493f408a991?d=retro&size=100"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[],"acceptLanguages":["en"],"organization":{"_id":"64467988902606605668dc8a","name":"MAPS-research","fullname":"MAPS Research","avatar":"https://www.gravatar.com/avatar/2df2a601ac42e11d30f6e493f408a991?d=retro&size=100"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.29387.md","query":{}}">
EvoGenUI-Bench: Evaluating LLMs as Multi-Turn Generative UI Assistants
Abstract
Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests evolve. We introduce EvoGenUI-Bench, a benchmark for multi-turn interface maintenance comprising 150 five-turn tasks and 750 turns across three scenarios: information presentation, executable interaction, and tool-grounded external state. We execute generated artifacts in a browser and evaluate them using screenshots, source and DOM evidence, actor traces, and runtime logs. Beyond turn-level and episode-level success, we measure cross-turn retention with Adjacent Pass Retention. Across eight models, even the strongest achieves 74.9% Turn Pass while completing only 37.3% of five-turn episodes; APR further falls to 52.4% on tool-grounded tasks. Diagnostic analysis shows that presentation failures center on information architecture, interaction failures on derived-state propagation and affordance binding, and tool-grounded failures additionally involve external-state grounding and requirement decomposition. These results reframe generative UI evaluation from judging isolated outputs to testing whether interface behavior, derived state, external state, and assistant claims remain synchronized as the artifact evolves.
Community
Can generative UI assistants keep one executable interface correct as user requirements evolve across multiple turns?
EvoGenUI-Bench introduces 150 five-turn tasks (750 turns) spanning information presentation, executable interaction, and tool-grounded external state. Generated artifacts are executed in a browser and evaluated using screenshots, source/DOM evidence, actor traces, and runtime logs.
Across eight models, the strongest reaches 74.9% Turn Pass but completes only 37.3% of full five-turn episodes. We also introduce Adjacent Pass Retention (APR) to measure whether behavior that worked at one turn survives the next update. The results show that cross-turn maintenance—not just one-shot visual quality—is a central bottleneck for reliable generative UI.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.29387 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.29387 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.29387 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.