SparsePR is a training-free sparse attention method for accelerating video generation. It combines response-coupled partitioning with probe-fitted residual reconstruction to reduce attention computation while preserving generation quality. We evaluate it across multiple video world models (HunyuanVideo, Wan2.2, Cosmos2.5, and Cosmos3-Nano) and provide an efficient block-sparse GPU implementation.</p>\n","updatedAt":"2026-08-24T13:56:46.423Z","author":{"_id":"65d60e4d15f94930d75f2e41","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/LISLywrxIHyfet7ds9ogq.png","fullname":"pardis","name":"PardisTaghavi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":0,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7979270815849304},"editors":["PardisTaghavi"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/LISLywrxIHyfet7ds9ogq.png"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.18484","authors":[{"_id":"6a8c4be98dd056518b7f529f","name":"Pardis Taghavi","hidden":false},{"_id":"6a8c4be98dd056518b7f52a0","name":"Reza Langari","hidden":false},{"_id":"6a8c4be98dd056518b7f52a1","name":"Gaurav Pandey","hidden":false}],"publishedAt":"2026-08-19T00:00:00.000Z","submittedOnDailyAt":"2026-08-24T00:00:00.000Z","title":"Partition the Support, Reconstruct the Residual: Training-Free Sparse Attention for Video Generation and World Models","submittedOnDailyBy":{"_id":"65d60e4d15f94930d75f2e41","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/LISLywrxIHyfet7ds9ogq.png","isPro":false,"fullname":"pardis","user":"PardisTaghavi","type":"user","name":"PardisTaghavi"},"summary":"Training-free block-sparse attention can accelerate video transformers, but row-wise attention concentration does not by itself specify an executable sparse operator. Queries sharing a block route may have poorly overlapping supports, while retained attention mass alone does not determine the post-softmax error from skipped interactions. We show that partition geometry affects both pooled support and the predictability of the remaining residual from the sparse output. We introduce SparsePR, which combines Response-Coupled Partitioning with Probe-Fitted Residual Reconstruction. Sampled-query key responses form paired K/V groups, whose centroids induce query-response coordinates for shared routing. A small set of exact query rows then calibrates a call-specific affine correction from the sparse output within the output subspace observed in the probe residuals. Across four heterogeneous video generation and world models, SparsePR consistently reduces attention-reconstruction error. Ablations show that probe fitting accounts for most of this reduction, while response-coupled partitioning lowers hard-drop error and improves reconstruction under a finite probe budget. SparsePR preserves generation quality at 22.0-26.0% realized executed-pair density while achieving 1.48x-2.61x end-to-end speedups. Project page: https://pardistaghavi.github.io/SparsePR-website/","upvotes":2,"discussionId":"6a8c4bea8dd056518b7f52a2","projectPage":"https://pardistaghavi.github.io/SparsePR-website/","githubRepo":"https://github.com/PardisTaghavi/SparsePR","githubRepoAddedBy":"user","ai_summary":"SparsePR accelerates video transformers via response-coupled partitioning and probe-fitted residual reconstruction, reducing attention error at low executed-pair densities with substantial speedups.","ai_keywords":["block-sparse attention","video transformers","SparsePR","Response-Coupled Partitioning","Probe-Fitted Residual Reconstruction","attention-reconstruction error","executed-pair density"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":14,"organization":{"_id":"693049768605dfa68334b46d","name":"TexasAMUniversity","fullname":"Texas A&M University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/68e396f2b5bb631e9b2fac9a/uv9z1cu15X7vyo70DW0tH.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"65d60e4d15f94930d75f2e41","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/LISLywrxIHyfet7ds9ogq.png","isPro":false,"fullname":"pardis","user":"PardisTaghavi","type":"user"},{"_id":"69af9fb149a53e7c237cb048","avatarUrl":"/avatars/f8f41cca34d396b33abd7c6f6ca09f1f.svg","isPro":false,"fullname":"Parnian","user":"ParnianTgh","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"693049768605dfa68334b46d","name":"TexasAMUniversity","fullname":"Texas A&M University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/68e396f2b5bb631e9b2fac9a/uv9z1cu15X7vyo70DW0tH.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.18484.md","query":{}}">
Partition the Support, Reconstruct the Residual: Training-Free Sparse Attention for Video Generation and World Models
Published on Aug 19
· Submitted by pardis on Aug 24 Abstract
SparsePR accelerates video transformers via response-coupled partitioning and probe-fitted residual reconstruction, reducing attention error at low executed-pair densities with substantial speedups.
Training-free block-sparse attention can accelerate video transformers, but row-wise attention concentration does not by itself specify an executable sparse operator. Queries sharing a block route may have poorly overlapping supports, while retained attention mass alone does not determine the post-softmax error from skipped interactions. We show that partition geometry affects both pooled support and the predictability of the remaining residual from the sparse output. We introduce SparsePR, which combines Response-Coupled Partitioning with Probe-Fitted Residual Reconstruction. Sampled-query key responses form paired K/V groups, whose centroids induce query-response coordinates for shared routing. A small set of exact query rows then calibrates a call-specific affine correction from the sparse output within the output subspace observed in the probe residuals. Across four heterogeneous video generation and world models, SparsePR consistently reduces attention-reconstruction error. Ablations show that probe fitting accounts for most of this reduction, while response-coupled partitioning lowers hard-drop error and improves reconstruction under a finite probe budget. SparsePR preserves generation quality at 22.0-26.0% realized executed-pair density while achieving 1.48x-2.61x end-to-end speedups. Project page: https://pardistaghavi.github.io/SparsePR-website/
Community
SparsePR is a training-free sparse attention method for accelerating video generation. It combines response-coupled partitioning with probe-fitted residual reconstruction to reduce attention computation while preserving generation quality. We evaluate it across multiple video world models (HunyuanVideo, Wan2.2, Cosmos2.5, and Cosmos3-Nano) and provide an efficient block-sparse GPU implementation.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.18484 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.18484 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.18484 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.