Hugging Face Daily Papers · July 1, 2026 · 6 min read

BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

Like Read original ↗

BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding Github: <a href=\"https://github.com/AMAP-ML/BlockPilot\" rel=\"nofollow\">https://github.com/AMAP-ML/BlockPilot</a>\n","updatedAt":"2026-07-01T03:03:49.579Z","author":{"_id":"64d1dc5273174cecdffc97d3","avatarUrl":"/avatars/6564e6b68fee9673f75b6366adf39a3b.svg","fullname":"Wang Yong","name":"seashell11","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.660621166229248},"editors":["seashell11"],"editorAvatarUrls":["/avatars/6564e6b68fee9673f75b6366adf39a3b.svg"],"reactions":[],"isReport":false}},{"id":"6a45c36a603dca9394434fe1","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":372,"isUserFollowing":false},"createdAt":"2026-07-02T01:48:26.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibration](https://huggingface.co/papers/2605.20022) (2026)\n* [PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding](https://huggingface.co/papers/2605.08632) (2026)\n* [Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting](https://huggingface.co/papers/2605.29727) (2026)\n* [Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding](https://huggingface.co/papers/2605.29707) (2026)\n* [DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding](https://huggingface.co/papers/2606.02091) (2026)\n* [Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing](https://huggingface.co/papers/2605.14978) (2026)\n* [D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting](https://huggingface.co/papers/2605.18810) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. \nThe following papers were recommended by the Semantic Scholar API \n<ul>\n<li><a href=\"https://huggingface.co/papers/2605.20022\">FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibration</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.08632\">PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.29727\">Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.29707\">Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.02091\">DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.14978\">Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.18810\">D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting</a> (2026)</li>\n</ul>\n Please give a thumbs up to this comment if you found it helpful!\n If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code>@librarian-bot recommend</code>\n","updatedAt":"2026-07-02T01:48:26.919Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":372,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7426207661628723},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.31315","authors":[{"_id":"6a44786041f04ae4d7ad96f4","name":"Hao Zhang","hidden":false},{"_id":"6a44786041f04ae4d7ad96f5","name":"Yiming Hu","hidden":false},{"_id":"6a44786041f04ae4d7ad96f6","name":"Yong Wang","hidden":false},{"_id":"6a44786041f04ae4d7ad96f7","name":"Mingqiao Mo","hidden":false},{"_id":"6a44786041f04ae4d7ad96f8","name":"Xin Xiao","hidden":false},{"_id":"6a44786041f04ae4d7ad96f9","name":"Xiangxiang Chu","hidden":false}],"publishedAt":"2026-06-30T00:00:00.000Z","submittedOnDailyAt":"2026-07-01T00:00:00.000Z","title":"BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding","submittedOnDailyBy":{"_id":"64d1dc5273174cecdffc97d3","avatarUrl":"/avatars/6564e6b68fee9673f75b6366adf39a3b.svg","isPro":false,"fullname":"Wang Yong","user":"seashell11","type":"user","name":"seashell11"},"summary":"Speculative decoding accelerates inference by using a lightweight draft model to generate candidate tokens in parallel, and are then verified by the target model, enabling lossless acceleration. Recently, diffusion-based speculative decoding further improves parallelism by generating multiple tokens per forward pass via block-level diffusion, achieving state-of-the-art (SOTA) performance. However, existing methods adopt a fixed inference block size and assume a uniform optimal decoding strategy across all inputs. In this paper, we show that this assumption is suboptimal, as the optimal block size varies across samples and plays a critical role in speculative decoding performance. Moreover, these values exhibit a clear local structure, concentrating around the training block size, which reduces the problem to a low-dimensional and structured decision space. Based on these insights, we propose BlockPilot, a sample-adaptive policy that predicts the optimal block size from the prefilling representation. Specifically, we formulate block size selection as a lightweight policy learning problem and propose an instance-adaptive decision mechanism that predicts the optimal block size based on the representation of the prefilling stage. The prediction is performed only once after prefilling, allowing for seamless integration. Extensive experiments demonstrate that our method is plug-and-play, introduces minimal overhead, and consistently improves efficiency, achieving an acceptance length of 5.92 and a 4.20times speedup on Qwen3-4B under temperature T=1.","upvotes":67,"discussionId":"6a44786041f04ae4d7ad96fa","githubRepo":"https://github.com/AMAP-ML/BlockPilot","githubRepoAddedBy":"user","ai_summary":"Speculative decoding with adaptive block size selection improves inference efficiency by predicting optimal block sizes from prefilling representations, achieving significant speedup with minimal overhead.","ai_keywords":["speculative decoding","draft model","target model","diffusion-based speculative decoding","block-level diffusion","inference block size","prefilling representation","policy learning","instance-adaptive decision mechanism"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":58},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64d1dc5273174cecdffc97d3","avatarUrl":"/avatars/6564e6b68fee9673f75b6366adf39a3b.svg","isPro":false,"fullname":"Wang Yong","user":"seashell11","type":"user"},{"_id":"69649c726a20f62862cf9578","avatarUrl":"/avatars/b5314840d74265d3fa4f88948146a729.svg","isPro":false,"fullname":"liukaikui","user":"liukaikui","type":"user"},{"_id":"661de9defdbc9c247f159d15","avatarUrl":"/avatars/38e21e78327cc908201122405c48f41b.svg","isPro":false,"fullname":"Rui Dai","user":"DerryD","type":"user"},{"_id":"65003db8bef9b594656f8fa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65003db8bef9b594656f8fa7/L6cvPOAeBRnFnIQwWxYyf.png","isPro":false,"fullname":"Hailang Huang","user":"lerogo","type":"user"},{"_id":"6773bcaa675a971ddf1e81dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/a8VUwZYXd7O_mq_zFvXMh.png","isPro":false,"fullname":"CokeWang","user":"CokeWang","type":"user"},{"_id":"663bbb61ec1aafe3d6c05558","avatarUrl":"/avatars/b2f593d0ae0adbaad9a7a99b490e3a2b.svg","isPro":false,"fullname":"Ziyu Ma","user":"poiuytrewq123","type":"user"},{"_id":"6459f2ae896f285ceb2384f0","avatarUrl":"/avatars/6bf4ed9ddd8f4dd45a97bec29274ae38.svg","isPro":false,"fullname":"wf","user":"Olivia0","type":"user"},{"_id":"68c261077e9a08a552ebcca4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/6G9CY69HxZZXAbAyueJ54.png","isPro":false,"fullname":"Bryan","user":"Bryan-A","type":"user"},{"_id":"62ec767dd4151c8b2c48f4aa","avatarUrl":"/avatars/a06e5715bdf222ca1911aec5d57a88c5.svg","isPro":false,"fullname":"Tianyu Zong","user":"UCASzty","type":"user"},{"_id":"66f3800367e707182cfabc92","avatarUrl":"/avatars/005c9ab9d7c78761c282c85a1258456d.svg","isPro":false,"fullname":"Haopeng Zhang","user":"espen123","type":"user"},{"_id":"660ae36c58941eba04907543","avatarUrl":"/avatars/0f59544d578d6dafa19492386551e9f2.svg","isPro":false,"fullname":"yuyongjia","user":"yyjok","type":"user"},{"_id":"64904c353be5db53615bd38a","avatarUrl":"/avatars/44296f0155fef0833aaf79201b5e344b.svg","isPro":false,"fullname":"chen zhihao","user":"mrbug","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.31315.md","query":{}}">

Papers

arxiv:2606.31315

BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding

Published on Jun 30

· Submitted by

Wang Yong on Jul 1

Upvote

Authors:

Abstract

Speculative decoding with adaptive block size selection improves inference efficiency by predicting optimal block sizes from prefilling representations, achieving significant speedup with minimal overhead.

Generated by Qwen/Qwen2.5-Coder-32B-Instruct

Speculative decoding accelerates inference by using a lightweight draft model to generate candidate tokens in parallel, and are then verified by the target model, enabling lossless acceleration. Recently, diffusion-based speculative decoding further improves parallelism by generating multiple tokens per forward pass via block-level diffusion, achieving state-of-the-art (SOTA) performance. However, existing methods adopt a fixed inference block size and assume a uniform optimal decoding strategy across all inputs. In this paper, we show that this assumption is suboptimal, as the optimal block size varies across samples and plays a critical role in speculative decoding performance. Moreover, these values exhibit a clear local structure, concentrating around the training block size, which reduces the problem to a low-dimensional and structured decision space. Based on these insights, we propose BlockPilot, a sample-adaptive policy that predicts the optimal block size from the prefilling representation. Specifically, we formulate block size selection as a lightweight policy learning problem and propose an instance-adaptive decision mechanism that predicts the optimal block size based on the representation of the prefilling stage. The prediction is performed only once after prefilling, allowing for seamless integration. Extensive experiments demonstrate that our method is plug-and-play, introduces minimal overhead, and consistently improves efficiency, achieving an acceptance length of 5.92 and a 4.20times speedup on Qwen3-4B under temperature T=1.