Hugging Face Daily Papers · · 6 min read

GameWAM: A World Action Model for Video Games

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

\n\t<a id=\"highlights\" class=\"block pr-1.5 text-lg md:absolute md:p-1.5 md:opacity-0 md:group-hover:opacity-100 md:right-full\" href=\"#highlights\" rel=\"nofollow\">\n\t\t<span class=\"header-link\"><svg class=\"text-gray-500 hover:text-black dark:hover:text-gray-200 w-4\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" aria-hidden=\"true\" role=\"img\" width=\"1em\" height=\"1em\" preserveAspectRatio=\"xMidYMid meet\" viewBox=\"0 0 256 256\"><path d=\"M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z\" fill=\"currentColor\"></path></svg></span>\n\t</a>\n\t<span>\n\t\tHighlights\n\t</span>\n</h2>\n<ul>\n<li><strong>A World Action Model for games:</strong> to our knowledge, GameWAM is the first WAM for native closed-loop gameplay and GUI control, jointly generating future visual observations and executable keyboard–mouse trajectories with parallel Video and Action DiTs, block-causal attention, and joint flow matching.</li>\n<li><strong>Unified heterogeneous native control:</strong> continuous camera or cursor motion and concurrent discrete controls share one Action-DiT flow, while a learned per-action router selects gameplay- or GUI-specific predictions and continuous-action normalization at every action timestep.</li>\n<li><strong>Long-horizon block–cycle control:</strong> each planning unit predicts beyond the committed horizon, executes only a short action prefix, and replans from new observations; a bounded within-cycle KV cache and hierarchical cross-cycle visual history preserve temporal continuity under bounded memory.</li>\n<li><strong>Competitive closed-loop performance:</strong> across Minecraft and ViZDoom, GameWAM achieves competitive task performance; in Minecraft, it uses fewer executed native actions than the compared agents across all evaluated task categories.</li>\n<li><strong>Low-Frequency Action Source Imprinting:</strong> GameWAM uncovers LASI, a source-sensitivity failure mode in which low-frequency components of the sampled action source steer coarse generated camera motion and can accumulate into persistent directional bias when reused across replanning steps.</li>\n</ul>\n","updatedAt":"2026-08-28T04:19:34.602Z","author":{"_id":"6877917cb2d7c0a48afb5487","avatarUrl":"/avatars/ef3ea56c925537a6ab745cea1a8b8067.svg","fullname":"Yuncheng Guo","name":"Yunncheng","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8458843231201172},"editors":["Yunncheng"],"editorAvatarUrls":["/avatars/ef3ea56c925537a6ab745cea1a8b8067.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.26200","authors":[{"_id":"6a9105dda64059bab69c3687","user":{"_id":"6877917cb2d7c0a48afb5487","avatarUrl":"/avatars/ef3ea56c925537a6ab745cea1a8b8067.svg","isPro":false,"fullname":"Yuncheng Guo","user":"Yunncheng","type":"user","name":"Yunncheng"},"name":"Yuncheng Guo","status":"claimed_verified","statusLastChangedAt":"2026-08-28T08:45:04.726Z","hidden":false},{"_id":"6a9105dda64059bab69c3688","name":"Zhanqiu Zhang","hidden":false},{"_id":"6a9105dda64059bab69c3689","name":"Yiwen Guo","hidden":false},{"_id":"6a9105dda64059bab69c368a","name":"Weijia Li","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/6877917cb2d7c0a48afb5487/O3uiisIaRVg2BkCGjW9-M.mp4"],"publishedAt":"2026-08-25T00:00:00.000Z","submittedOnDailyAt":"2026-08-28T00:00:00.000Z","title":"GameWAM: A World Action Model for Video Games","submittedOnDailyBy":{"_id":"6877917cb2d7c0a48afb5487","avatarUrl":"/avatars/ef3ea56c925537a6ab745cea1a8b8067.svg","isPro":false,"fullname":"Yuncheng Guo","user":"Yunncheng","type":"user","name":"Yunncheng"},"summary":"Modern video games combine first-person perception, rapid visual changes, persistent world state, and heterogeneous native controls. Existing game agents map visual and task context directly to actions but lack explicit world dynamics modeling, whereas interactive game world models predict visual futures from supplied actions but do not serve as task policies. World-Action Models (WAMs) unify these objectives, but remain largely unexplored under the dynamics and open-ended interaction of video games. We introduce GameWAM, to our knowledge the first WAM for native closed-loop gameplay and GUI control. GameWAM jointly generates future visual observations and executable keyboard-mouse trajectories through parallel visual and action generative processes with block-causal conditioning and flow matching. To support joint world-action learning, we construct synchronized gameplay and GUI trajectories. To handle heterogeneous native control, GameWAM predicts a gameplay/GUI mode at each action step and generates actions with mode-specific prediction distributions and continuous-action normalization. For long-horizon interaction, block-cycle control predicts beyond the committed horizon, executes only a short action prefix, and replans from new observations, while fine-grained within-cycle context and hierarchical cross-cycle history preserve temporal continuity. Experiments demonstrate competitive task success with fewer executed native actions than the compared agents. We further uncover Low-Frequency Action Source Imprinting (LASI), in which low-frequency components of the sampled action source systematically steer coarse generated camera motion under fixed conditioning, revealing a source-sensitivity failure mode in generative control. Project page is available at https://yunncheng.github.io/GameWAM/.","upvotes":17,"discussionId":"6a9105dda64059bab69c368b","projectPage":"https://yunncheng.github.io/GameWAM/","githubRepo":"https://github.com/yunncheng/GameWAM","githubRepoAddedBy":"user","ai_summary":"GameWAM is a unified world-action model for native video-game control that jointly predicts future visuals and executable keyboard-mouse actions using block-causal flow matching, mode-specific distributions, and block-cycle replanning.","ai_keywords":["World-Action Models","flow matching","block-causal conditioning","block-cycle control","mode-specific prediction distributions","continuous-action normalization","Low-Frequency Action Source Imprinting","generative control"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":1,"organization":{"_id":"66543b6e420092799d2f625c","name":"tencent","fullname":"Tencent","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/Lp3m-XLpjQGwBItlvn69q.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6877917cb2d7c0a48afb5487","avatarUrl":"/avatars/ef3ea56c925537a6ab745cea1a8b8067.svg","isPro":false,"fullname":"Yuncheng Guo","user":"Yunncheng","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"6487e158f675b4a7867f45fa","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6487e158f675b4a7867f45fa/J0sls6zZ682o-SH7iQs7B.jpeg","isPro":false,"fullname":"Zilong Huang","user":"SereinH","type":"user"},{"_id":"62728e2662cc13e9ef4f6b72","avatarUrl":"/avatars/0f3144e4cfb247faeeb84fc70d10bdbb.svg","isPro":false,"fullname":"QQQ","user":"QAQqaq","type":"user"},{"_id":"664d930f4b870dd167473c1c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/664d930f4b870dd167473c1c/TXVEPGvkhftdI_xE1mluu.jpeg","isPro":false,"fullname":"Andy Guan","user":"andytonglove","type":"user"},{"_id":"69366520d3108f12cd6f6fb0","avatarUrl":"/avatars/ddbce09d436895ae1bdde1dc87bb0d85.svg","isPro":false,"fullname":"Zecong Tang","user":"Zecongloveai","type":"user"},{"_id":"687f853bb39262ba84f3eeff","avatarUrl":"/avatars/cdfc44fde8237f08f10192553fe5a075.svg","isPro":false,"fullname":"Junhao Shen","user":"shenjunhao","type":"user"},{"_id":"692f9daa20092e3c4123cd01","avatarUrl":"/avatars/803155f02bae7a099310d7c47016406c.svg","isPro":false,"fullname":"Irene","user":"irenehere","type":"user"},{"_id":"6355473d525beaee688b7ba1","avatarUrl":"/avatars/1fb0d57ed5f1a9b872a1ada8b2973ffb.svg","isPro":false,"fullname":"Wei Tao","user":"itaowe","type":"user"},{"_id":"69afd11e962a5ad4e31c460a","avatarUrl":"/avatars/fe2089ad472e2d69d271a86643973d6d.svg","isPro":false,"fullname":"YINGGUANG YANG","user":"younginglights","type":"user"},{"_id":"6819b3bfb946b69d0dd80e8c","avatarUrl":"/avatars/96e06eb2880f121295da6d88a520a5cd.svg","isPro":false,"fullname":"JoeK","user":"JoeKKKK","type":"user"},{"_id":"690c35c8dd10faadd5ae2f80","avatarUrl":"/avatars/993d7d59f7a0b070306bb10a97cb7c35.svg","isPro":true,"fullname":"Sungwoo Park","user":"swpark5","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"66543b6e420092799d2f625c","name":"tencent","fullname":"Tencent","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/Lp3m-XLpjQGwBItlvn69q.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.26200.md","query":{}}">
Papers
arxiv:2608.26200

GameWAM: A World Action Model for Video Games

Published on Aug 25
· Submitted by
Yuncheng Guo
on Aug 28
Authors:

Abstract

GameWAM is a unified world-action model for native video-game control that jointly predicts future visuals and executable keyboard-mouse actions using block-causal flow matching, mode-specific distributions, and block-cycle replanning.

Modern video games combine first-person perception, rapid visual changes, persistent world state, and heterogeneous native controls. Existing game agents map visual and task context directly to actions but lack explicit world dynamics modeling, whereas interactive game world models predict visual futures from supplied actions but do not serve as task policies. World-Action Models (WAMs) unify these objectives, but remain largely unexplored under the dynamics and open-ended interaction of video games. We introduce GameWAM, to our knowledge the first WAM for native closed-loop gameplay and GUI control. GameWAM jointly generates future visual observations and executable keyboard-mouse trajectories through parallel visual and action generative processes with block-causal conditioning and flow matching. To support joint world-action learning, we construct synchronized gameplay and GUI trajectories. To handle heterogeneous native control, GameWAM predicts a gameplay/GUI mode at each action step and generates actions with mode-specific prediction distributions and continuous-action normalization. For long-horizon interaction, block-cycle control predicts beyond the committed horizon, executes only a short action prefix, and replans from new observations, while fine-grained within-cycle context and hierarchical cross-cycle history preserve temporal continuity. Experiments demonstrate competitive task success with fewer executed native actions than the compared agents. We further uncover Low-Frequency Action Source Imprinting (LASI), in which low-frequency components of the sampled action source systematically steer coarse generated camera motion under fixed conditioning, revealing a source-sensitivity failure mode in generative control. Project page is available at https://yunncheng.github.io/GameWAM/.

Community

Paper author Paper submitter about 6 hours ago

Highlights

  • A World Action Model for games: to our knowledge, GameWAM is the first WAM for native closed-loop gameplay and GUI control, jointly generating future visual observations and executable keyboard–mouse trajectories with parallel Video and Action DiTs, block-causal attention, and joint flow matching.
  • Unified heterogeneous native control: continuous camera or cursor motion and concurrent discrete controls share one Action-DiT flow, while a learned per-action router selects gameplay- or GUI-specific predictions and continuous-action normalization at every action timestep.
  • Long-horizon block–cycle control: each planning unit predicts beyond the committed horizon, executes only a short action prefix, and replans from new observations; a bounded within-cycle KV cache and hierarchical cross-cycle visual history preserve temporal continuity under bounded memory.
  • Competitive closed-loop performance: across Minecraft and ViZDoom, GameWAM achieves competitive task performance; in Minecraft, it uses fewer executed native actions than the compared agents across all evaluated task categories.
  • Low-Frequency Action Source Imprinting: GameWAM uncovers LASI, a source-sensitivity failure mode in which low-frequency components of the sampled action source steer coarse generated camera motion and can accumulate into persistent directional bias when reused across replanning steps.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images

· Sign up or log in to comment

Get this paper in your agent:

hf papers read 2608.26200
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper

No model linking this paper

Cite arxiv.org/abs/2608.26200 in a model README.md to link it from this page.

Datasets citing this paper

No dataset linking this paper

Cite arxiv.org/abs/2608.26200 in a dataset README.md to link it from this page.

Spaces citing this paper

No Space linking this paper

Cite arxiv.org/abs/2608.26200 in a Space README.md to link it from this page.

Collections including this paper

No Collection including this paper

Add this paper to a collection to link it from this page.

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from Hugging Face Daily Papers