Vision-Language-Action (VLA) models enable instruction-driven robotic manipulation, but they inherit oversized language backbones from pretrained VLMs whose capacity far exceeds what is needed for short robotic instructions. This raises a basic question: how much of a VLA model is actually necessary for closed-loop control? In this work, we study architectural redundancy in VLA models by using transformer block removal as a controlled intervention. We introduce \\textbf{Drop-Then-Recovery (DTR)}, an analysis protocol that removes selected blocks from a pretrained VLA model and then fine-tunes the resulting model to measure whether the removed capacity was necessary for downstream control. To make this intervention reliable, we propose \\textbf{GateProbe}, a one-shot virtual-gate sensitivity metric that ranks blocks by their contribution to the downstream action loss. Across multiple VLA architectures, manipulation benchmarks and even real-robot industrial scenarios, we find a strong asymmetry in post-removal recoverability: \\ul{\\textit{language backbones are highly redundant for standard robotic manipulation tasks, whereas vision and action pathways are substantially less tolerant to removal}}. On LIBERO, removing half of the LLM blocks even improves OpenVLA-OFT from 95.0% to 98.3% under the same downstream fine-tuning budget, and retaining only two language blocks still recovers baseline-level performance. These results suggest that current VLA benchmarks may exert limited pressure on deep language grounding and compositional instruction understanding, and that future VLA architectures should allocate capacity more deliberately across language, vision, and action components. The code is available at this https URL.</p>\n","updatedAt":"2026-06-30T20:25:55.874Z","author":{"_id":"6438f5555aa69077ffb16218","avatarUrl":"/avatars/582f736c0485252010b682e85db62e77.svg","fullname":"Guoheng Sun","name":"s1ghhh","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":9,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8720217943191528},"editors":["s1ghhh"],"editorAvatarUrls":["/avatars/582f736c0485252010b682e85db62e77.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.27755","authors":[{"_id":"6a4424d141f04ae4d7ad95f4","name":"Guoheng Sun","hidden":false},{"_id":"6a4424d141f04ae4d7ad95f5","name":"Kaixi Feng","hidden":false},{"_id":"6a4424d141f04ae4d7ad95f6","name":"Shwai He","hidden":false},{"_id":"6a4424d141f04ae4d7ad95f7","name":"Xiaochuan Gong","hidden":false},{"_id":"6a4424d141f04ae4d7ad95f8","name":"Yexiao He","hidden":false},{"_id":"6a4424d141f04ae4d7ad95f9","name":"Ziyao Wang","hidden":false},{"_id":"6a4424d141f04ae4d7ad95fa","name":"Zheyu Shen","hidden":false},{"_id":"6a4424d141f04ae4d7ad95fb","name":"Wanghao Ye","hidden":false},{"_id":"6a4424d141f04ae4d7ad95fc","name":"Ramana Rao Kompella","hidden":false},{"_id":"6a4424d141f04ae4d7ad95fd","name":"Gaowen Liu","hidden":false},{"_id":"6a4424d141f04ae4d7ad95fe","name":"Ang Li","hidden":false}],"publishedAt":"2026-06-26T00:00:00.000Z","submittedOnDailyAt":"2026-06-30T00:00:00.000Z","title":"Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?","submittedOnDailyBy":{"_id":"6438f5555aa69077ffb16218","avatarUrl":"/avatars/582f736c0485252010b682e85db62e77.svg","isPro":false,"fullname":"Guoheng Sun","user":"s1ghhh","type":"user","name":"s1ghhh"},"summary":"Vision-Language-Action (VLA) models enable instruction-driven robotic manipulation, but they inherit oversized language backbones from pretrained VLMs whose capacity far exceeds what is needed for short robotic instructions. This raises a basic question: how much of a VLA model is actually necessary for closed-loop control? In this work, we study architectural redundancy in VLA models by using transformer block removal as a controlled intervention. We introduce Drop-Then-Recovery (DTR), an analysis protocol that removes selected blocks from a pretrained VLA model and then fine-tunes the resulting model to measure whether the removed capacity was necessary for downstream control. To make this intervention reliable, we propose GateProbe, a one-shot virtual-gate sensitivity metric that ranks blocks by their contribution to the downstream action loss. Across multiple VLA architectures, manipulation benchmarks and even real-robot industrial scenarios, we find a strong asymmetry in post-removal recoverability: \\textit{language backbones are highly redundant for standard robotic manipulation tasks, whereas vision and action pathways are substantially less tolerant to removal}. On LIBERO, removing half of the LLM blocks even improves OpenVLA-OFT from 95.0% to 98.3% under the same downstream fine-tuning budget, and retaining only two language blocks still recovers baseline-level performance. These results suggest that current VLA benchmarks may exert limited pressure on deep language grounding and compositional instruction understanding, and that future VLA architectures should allocate capacity more deliberately across language, vision, and action components. The code is available at https://github.com/s1ghhh/VLADrop.","upvotes":2,"discussionId":"6a4424d241f04ae4d7ad95ff","githubRepo":"https://github.com/s1ghhh/VLADrop","githubRepoAddedBy":"user","ai_summary":"Research reveals that language backbones in Vision-Language-Action models are highly redundant for robotic manipulation tasks, while vision and action pathways are more critical, suggesting need for deliberate capacity allocation in future architectures.","ai_keywords":["Vision-Language-Action models","transformer block removal","Drop-Then-Recovery","GateProbe","language backbones","vision pathways","action pathways","OpenVLA-OFT","LIBERO","compositional instruction understanding"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":0,"organization":{"_id":"66dde4e47ec0e5f42133d606","name":"LLM-Drop","fullname":"LLM-Drop","avatar":"https://www.gravatar.com/avatar/d575c2fca710ed421d13c674278bcabe?d=retro&size=100"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6438f5555aa69077ffb16218","avatarUrl":"/avatars/582f736c0485252010b682e85db62e77.svg","isPro":false,"fullname":"Guoheng Sun","user":"s1ghhh","type":"user"},{"_id":"64468d7a3b28db22e7f84418","avatarUrl":"/avatars/ff05994acd7da64a2ed6e44f12cf77bc.svg","isPro":false,"fullname":"Shwai He","user":"shwai-he","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"66dde4e47ec0e5f42133d606","name":"LLM-Drop","fullname":"LLM-Drop","avatar":"https://www.gravatar.com/avatar/d575c2fca710ed421d13c674278bcabe?d=retro&size=100"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.27755.md","query":{}}">
Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?
Authors: ,
,
,
,
,
,
,
,
,
,
Abstract
Research reveals that language backbones in Vision-Language-Action models are highly redundant for robotic manipulation tasks, while vision and action pathways are more critical, suggesting need for deliberate capacity allocation in future architectures.
Vision-Language-Action (VLA) models enable instruction-driven robotic manipulation, but they inherit oversized language backbones from pretrained VLMs whose capacity far exceeds what is needed for short robotic instructions. This raises a basic question: how much of a VLA model is actually necessary for closed-loop control? In this work, we study architectural redundancy in VLA models by using transformer block removal as a controlled intervention. We introduce Drop-Then-Recovery (DTR), an analysis protocol that removes selected blocks from a pretrained VLA model and then fine-tunes the resulting model to measure whether the removed capacity was necessary for downstream control. To make this intervention reliable, we propose GateProbe, a one-shot virtual-gate sensitivity metric that ranks blocks by their contribution to the downstream action loss. Across multiple VLA architectures, manipulation benchmarks and even real-robot industrial scenarios, we find a strong asymmetry in post-removal recoverability: \textit{language backbones are highly redundant for standard robotic manipulation tasks, whereas vision and action pathways are substantially less tolerant to removal}. On LIBERO, removing half of the LLM blocks even improves OpenVLA-OFT from 95.0% to 98.3% under the same downstream fine-tuning budget, and retaining only two language blocks still recovers baseline-level performance. These results suggest that current VLA benchmarks may exert limited pressure on deep language grounding and compositional instruction understanding, and that future VLA architectures should allocate capacity more deliberately across language, vision, and action components. The code is available at https://github.com/s1ghhh/VLADrop.
Community
Vision-Language-Action (VLA) models enable instruction-driven robotic manipulation, but they inherit oversized language backbones from pretrained VLMs whose capacity far exceeds what is needed for short robotic instructions. This raises a basic question: how much of a VLA model is actually necessary for closed-loop control? In this work, we study architectural redundancy in VLA models by using transformer block removal as a controlled intervention. We introduce \textbf{Drop-Then-Recovery (DTR)}, an analysis protocol that removes selected blocks from a pretrained VLA model and then fine-tunes the resulting model to measure whether the removed capacity was necessary for downstream control. To make this intervention reliable, we propose \textbf{GateProbe}, a one-shot virtual-gate sensitivity metric that ranks blocks by their contribution to the downstream action loss. Across multiple VLA architectures, manipulation benchmarks and even real-robot industrial scenarios, we find a strong asymmetry in post-removal recoverability: \ul{\textit{language backbones are highly redundant for standard robotic manipulation tasks, whereas vision and action pathways are substantially less tolerant to removal}}. On LIBERO, removing half of the LLM blocks even improves OpenVLA-OFT from 95.0% to 98.3% under the same downstream fine-tuning budget, and retaining only two language blocks still recovers baseline-level performance. These results suggest that current VLA benchmarks may exert limited pressure on deep language grounding and compositional instruction understanding, and that future VLA architectures should allocate capacity more deliberately across language, vision, and action components. The code is available at this https URL.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.27755 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.27755 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.27755 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.