Hugging Face Daily Papers · September 1, 2026 · 6 min read

CoVA-SFT: A Large-Scale Dataset for Chain of Visual Abstractions

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

Like Read original ↗

What if thinking itself could be visual, even when the problem isn't?\nWe release CoVA-SFT: 51.9K visual reasoning traces (222K rendered steps) for text-only problems across 17 tasks, plus CoVA-Bench.\n📍 EMNLP 2026 Findings 📄 <a href=\"https://arxiv.org/abs/2608.28958\" rel=\"nofollow\">https://arxiv.org/abs/2608.28958</a> 🤗 <a href=\"https://huggingface.co/datasets/tsunghanwu/cova\">https://huggingface.co/datasets/tsunghanwu/cova</a> <a href=\"https://cdn-uploads.huggingface.co/production/uploads/644a767044b75fd95805d232/DQ1QZOqCuoWWfc7l2ruM8.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/644a767044b75fd95805d232/DQ1QZOqCuoWWfc7l2ruM8.png\" alt=\"fig2_X\"></a>\n","updatedAt":"2026-09-01T17:52:34.381Z","author":{"_id":"644a767044b75fd95805d232","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/644a767044b75fd95805d232/vHA2vI_B3CpXapdBEwspB.jpeg","fullname":"Patrick (Tsung-Han) Wu","name":"tsunghanwu","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":9,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7299033999443054},"editors":["tsunghanwu"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/644a767044b75fd95805d232/vHA2vI_B3CpXapdBEwspB.jpeg"],"reactions":[],"isReport":false}},{"id":"6a977acfe282fe351baf75c0","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":379,"isUserFollowing":false},"createdAt":"2026-09-02T01:24:31.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [ChronoVision: Temporal Reasoning via Latent State Reconstruction](https://huggingface.co/papers/2608.05631) (2026)\n* [GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models](https://huggingface.co/papers/2608.06769) (2026)\n* [GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning](https://huggingface.co/papers/2608.15517) (2026)\n* [VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning](https://huggingface.co/papers/2608.26105) (2026)\n* [ComplexityWorld: Benchmarking Vision-Language Models on Verifiable Visual Decision Making](https://huggingface.co/papers/2608.07584) (2026)\n* [Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning](https://huggingface.co/papers/2608.15869) (2026)\n* [Search, Fail, Recover: A Training Framework for Correction-Aware Reasoning](https://huggingface.co/papers/2607.07492) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. \nThe following papers were recommended by the Semantic Scholar API \n<ul>\n<li><a href=\"https://huggingface.co/papers/2608.05631\">ChronoVision: Temporal Reasoning via Latent State Reconstruction</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2608.06769\">GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2608.15517\">GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2608.26105\">VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2608.07584\">ComplexityWorld: Benchmarking Vision-Language Models on Verifiable Visual Decision Making</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2608.15869\">Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.07492\">Search, Fail, Recover: A Training Framework for Correction-Aware Reasoning</a> (2026)</li>\n</ul>\n Please give a thumbs up to this comment if you found it helpful!\n If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code>@librarian-bot recommend</code>\n","updatedAt":"2026-09-02T01:24:31.799Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":379,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7071583271026611},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.28958","authors":[{"_id":"6a971086fe3c2f89286c370c","name":"Tsung-Han Wu","hidden":false},{"_id":"6a971086fe3c2f89286c370d","user":{"_id":"677c834f3acf9d7d141df32a","avatarUrl":"/avatars/34f3ed50a1cb797213519f9a0eeafecc.svg","isPro":false,"fullname":"Heekyung Lee","user":"Kyunnilee","type":"user","name":"Kyunnilee"},"name":"Heekyung Lee","status":"claimed_verified","statusLastChangedAt":"2026-09-02T00:45:04.517Z","hidden":false},{"_id":"6a971086fe3c2f89286c370e","user":{"_id":"637c26aaa8716d64204ec799","avatarUrl":"/avatars/58071290c4a0bca3a9d37cf4d841bf62.svg","isPro":false,"fullname":"Anya Ji","user":"anya-ji","type":"user","name":"anya-ji"},"name":"Anya Ji","status":"claimed_verified","statusLastChangedAt":"2026-09-02T00:45:04.533Z","hidden":false},{"_id":"6a971086fe3c2f89286c370f","user":{"_id":"685c9a271fb286e47dc99eca","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/x7804Zzt1jlSFjeh0T2nE.png","isPro":false,"fullname":"Haoming Chen","user":"haoming06","type":"user","name":"haoming06"},"name":"Haoming Chen","status":"claimed_verified","statusLastChangedAt":"2026-09-02T00:45:04.540Z","hidden":false},{"_id":"6a971086fe3c2f89286c3710","name":"Trevor Darrell","hidden":false},{"_id":"6a971086fe3c2f89286c3711","name":"Joseph E. Gonzalez","hidden":false},{"_id":"6a971086fe3c2f89286c3712","user":{"_id":"6388f68c43d8b0797a09ff84","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1669920369989-noauth.jpeg","isPro":false,"fullname":"David Chan","user":"davidchan","type":"user","name":"davidchan"},"name":"David M. Chan","status":"claimed_verified","statusLastChangedAt":"2026-09-02T00:45:04.525Z","hidden":false}],"publishedAt":"2026-08-29T00:00:00.000Z","submittedOnDailyAt":"2026-09-01T00:00:00.000Z","title":"CoVA-SFT: A Large-Scale Dataset for Chain of Visual Abstractions","submittedOnDailyBy":{"_id":"644a767044b75fd95805d232","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/644a767044b75fd95805d232/vHA2vI_B3CpXapdBEwspB.jpeg","isPro":false,"fullname":"Patrick (Tsung-Han) Wu","user":"tsunghanwu","type":"user","name":"tsunghanwu"},"summary":"Chain-of-thought (CoT) reasoning has dramatically improved large language models (LLMs) by allowing them to decompose problems into intermediate steps. While CoT is widely effective for linguistic tasks, text-only CoT forces models to serialize visual problems into awkward prose. Although architectural solutions exist to process visual inputs, the community lacks a massive, multi-step, self-corrected dataset to teach models how to build and maintain internal visual workspaces when solving purely textual reasoning problems. To address this limitation, we introduce CoVA-SFT, a highly structured corpus of 51.9K samples containing over 222K multimodal reasoning steps across 5 distinct layout families and 17 complex tasks, and CoVA-Bench, a companion benchmark of 1,700 held-out test samples spanning the same tasks for reproducible evaluation. By providing explicit rationale formulations, agentic renderings, and verification loops, CoVA-SFT teaches multimodal language models to interleave text and visual abstractions. We validate the dataset by demonstrating that models fine-tuned on CoVA-SFT outperform all interleaved CoT baselines by more than 2x on average on CoVA-Bench, though they still fall short of strong text-only CoT baselines, highlighting open challenges for future work.","upvotes":4,"discussionId":"6a971087fe3c2f89286c3713","ai_summary":"CoVA-SFT is a multimodal reasoning dataset that teaches language models to interleave text and visual abstractions via structured reasoning steps, improving performance on visual reasoning benchmarks.","ai_keywords":["chain-of-thought reasoning","multimodal reasoning","visual workspaces","CoVA-SFT","CoVA-Bench","agentic renderings","verification loops","interleaved CoT"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"66b1baeff10262fc4fa61961","name":"UCBerkeley","fullname":"University of California, Berkeley","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/63f425c3a096536aeab42dea/bxNKEkprdm5JI1wkjmNAL.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"677c834f3acf9d7d141df32a","avatarUrl":"/avatars/34f3ed50a1cb797213519f9a0eeafecc.svg","isPro":false,"fullname":"Heekyung Lee","user":"Kyunnilee","type":"user"},{"_id":"644a767044b75fd95805d232","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/644a767044b75fd95805d232/vHA2vI_B3CpXapdBEwspB.jpeg","isPro":false,"fullname":"Patrick (Tsung-Han) Wu","user":"tsunghanwu","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"684d57f26e04c265777ead3f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/cuOj-bQqukSZreXgUJlfm.png","isPro":false,"fullname":"Joakim Lee","user":"Reinforcement4All","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"66b1baeff10262fc4fa61961","name":"UCBerkeley","fullname":"University of California, Berkeley","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/63f425c3a096536aeab42dea/bxNKEkprdm5JI1wkjmNAL.png"},"query":{}}">

Papers

arxiv:2608.28958

CoVA-SFT: A Large-Scale Dataset for Chain of Visual Abstractions

Published on Aug 29

· Submitted by

Patrick (Tsung-Han) Wu on Sep 1

University of California, Berkeley

Upvote

Authors:

Heekyung Lee ,

Anya Ji ,

Haoming Chen ,

David M. Chan

Abstract

CoVA-SFT is a multimodal reasoning dataset that teaches language models to interleave text and visual abstractions via structured reasoning steps, improving performance on visual reasoning benchmarks.

Generated by thinkingmachines/Inkling-Small

Chain-of-thought (CoT) reasoning has dramatically improved large language models (LLMs) by allowing them to decompose problems into intermediate steps. While CoT is widely effective for linguistic tasks, text-only CoT forces models to serialize visual problems into awkward prose. Although architectural solutions exist to process visual inputs, the community lacks a massive, multi-step, self-corrected dataset to teach models how to build and maintain internal visual workspaces when solving purely textual reasoning problems. To address this limitation, we introduce CoVA-SFT, a highly structured corpus of 51.9K samples containing over 222K multimodal reasoning steps across 5 distinct layout families and 17 complex tasks, and CoVA-Bench, a companion benchmark of 1,700 held-out test samples spanning the same tasks for reproducible evaluation. By providing explicit rationale formulations, agentic renderings, and verification loops, CoVA-SFT teaches multimodal language models to interleave text and visual abstractions. We validate the dataset by demonstrating that models fine-tuned on CoVA-SFT outperform all interleaved CoT baselines by more than 2x on average on CoVA-Bench, though they still fall short of strong text-only CoT baselines, highlighting open challenges for future work.