A Diffusion MoE Framework with Saliency-Harnessing Accurate Routing</p>\n","updatedAt":"2026-06-30T02:41:04.177Z","author":{"_id":"665fce33b99c631f4f57e650","avatarUrl":"/avatars/dc98235d6e4e33e5980c2b46627c238b.svg","fullname":"Haoyou Deng","name":"haoyou11","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8054758906364441},"editors":["haoyou11"],"editorAvatarUrls":["/avatars/dc98235d6e4e33e5980c2b46627c238b.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.26938","authors":[{"_id":"6a432c4f763f63ca3757e84d","name":"Haoyou Deng","hidden":false},{"_id":"6a432c4f763f63ca3757e84e","name":"Keyu Yan","hidden":false},{"_id":"6a432c4f763f63ca3757e84f","name":"Chaojie Mao","hidden":false},{"_id":"6a432c4f763f63ca3757e850","name":"Xiang Wang","hidden":false},{"_id":"6a432c4f763f63ca3757e851","name":"Yu Liu","hidden":false},{"_id":"6a432c4f763f63ca3757e852","name":"Changxin Gao","hidden":false},{"_id":"6a432c4f763f63ca3757e853","name":"Nong Sang","hidden":false}],"publishedAt":"2026-06-25T00:00:00.000Z","submittedOnDailyAt":"2026-06-30T00:00:00.000Z","title":"Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoE","submittedOnDailyBy":{"_id":"665fce33b99c631f4f57e650","avatarUrl":"/avatars/dc98235d6e4e33e5980c2b46627c238b.svg","isPro":false,"fullname":"Haoyou Deng","user":"haoyou11","type":"user","name":"haoyou11"},"summary":"Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling diffusion models in visual generation. Recent advancements have focused on adaptively allocating computational resources across diverse tokens to improve efficiency and performance. However, we identify a routing assignment problem in existing diffusion MoE frameworks: the router fails to accurately allocate more computational resources to salient tokens. Our analysis attributes this failure to the router's reliance on noise-corrupted latent features throughout the denoising process. Such stochastic noise obscures the critical structural and textural information, thereby preventing the router from effectively distinguishing salient tokens. To address this, we propose SharpMoE, a post-training framework with a saliency-harnessing accurate routing mechanism, which utilizes clean latent features as a noise-free guidance signal for routing. By bypassing the noise-distorted inputs, SharpMoE provides the router with clear saliency guidance, enabling the identification of salient tokens even in high-noise stages. Furthermore, we introduce a trajectory routing loss to constrain the compute allocation throughout the multi-step denoising trajectory, ensuring precise resource allocation along the generation rollout. Extensive experiments demonstrate that SharpMoE serves as a versatile, plug-and-play solution that further enhances the pretrained, converged MoE models, achieving state-of-the-art performance in visual generation.","upvotes":3,"discussionId":"6a432c50763f63ca3757e854","ai_summary":"SharpMoE addresses routing inefficiencies in diffusion models by using clean latent features to guide salient token identification and employs trajectory routing loss for precise compute allocation during multi-step denoising.","ai_keywords":["Mixture-of-Experts","diffusion models","routing assignment","latent features","denoising process","salient tokens","post-training framework","trajectory routing loss","compute allocation","visual generation"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","organization":{"_id":"67d15cca6e2cf0e062dbfb54","name":"AlibabaTongyiLab","fullname":"TongyiLab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/67d1502bfabfe9974d1f77bb/XdUSVf6HqBzE7zFBfSDQP.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"665fce33b99c631f4f57e650","avatarUrl":"/avatars/dc98235d6e4e33e5980c2b46627c238b.svg","isPro":false,"fullname":"Haoyou Deng","user":"haoyou11","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"67d15cca6e2cf0e062dbfb54","name":"AlibabaTongyiLab","fullname":"TongyiLab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/67d1502bfabfe9974d1f77bb/XdUSVf6HqBzE7zFBfSDQP.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.26938.md","query":{}}">
Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoE
Abstract
SharpMoE addresses routing inefficiencies in diffusion models by using clean latent features to guide salient token identification and employs trajectory routing loss for precise compute allocation during multi-step denoising.
Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling diffusion models in visual generation. Recent advancements have focused on adaptively allocating computational resources across diverse tokens to improve efficiency and performance. However, we identify a routing assignment problem in existing diffusion MoE frameworks: the router fails to accurately allocate more computational resources to salient tokens. Our analysis attributes this failure to the router's reliance on noise-corrupted latent features throughout the denoising process. Such stochastic noise obscures the critical structural and textural information, thereby preventing the router from effectively distinguishing salient tokens. To address this, we propose SharpMoE, a post-training framework with a saliency-harnessing accurate routing mechanism, which utilizes clean latent features as a noise-free guidance signal for routing. By bypassing the noise-distorted inputs, SharpMoE provides the router with clear saliency guidance, enabling the identification of salient tokens even in high-noise stages. Furthermore, we introduce a trajectory routing loss to constrain the compute allocation throughout the multi-step denoising trajectory, ensuring precise resource allocation along the generation rollout. Extensive experiments demonstrate that SharpMoE serves as a versatile, plug-and-play solution that further enhances the pretrained, converged MoE models, achieving state-of-the-art performance in visual generation.
Community
A Diffusion MoE Framework with Saliency-Harnessing Accurate Routing
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.26938 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.26938 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.26938 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.