Hugging Face Daily Papers · September 9, 2026 · 3 min read

VidaForge: Open Research Infrastructure for Video Pretraining Data Recipes

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

Like Read original ↗

code: <a href=\"https://github.com/GAIR-NLP/VidaForge\" rel=\"nofollow\">https://github.com/GAIR-NLP/VidaForge</a><br>paper: <a href=\"https://arxiv.org/pdf/2609.06652\" rel=\"nofollow\">https://arxiv.org/pdf/2609.06652</a></p>\n","updatedAt":"2026-09-09T03:22:22.313Z","author":{"_id":"633fc70529b5a95f6e15a6b7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/Fzh7wWuqU-fBbzdupOUtF.jpeg","fullname":"Yan Ma","name":"ManTle","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":7,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6150314211845398},"editors":["ManTle"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/Fzh7wWuqU-fBbzdupOUtF.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2609.06652","authors":[{"_id":"6aa0d0bcd0174964227becdf","user":{"_id":"633fc70529b5a95f6e15a6b7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/Fzh7wWuqU-fBbzdupOUtF.jpeg","isPro":false,"fullname":"Yan Ma","user":"ManTle","type":"user","name":"ManTle"},"name":"Yan Ma","status":"claimed_verified","statusLastChangedAt":"2026-09-09T08:45:04.609Z","hidden":false},{"_id":"6aa0d0bcd0174964227bece0","name":"Jiadi Su","hidden":false},{"_id":"6aa0d0bcd0174964227bece1","name":"Zhulin Hu","hidden":false},{"_id":"6aa0d0bcd0174964227bece2","name":"Ethan Chern","hidden":false},{"_id":"6aa0d0bcd0174964227bece3","name":"Linhao Zhang","hidden":false},{"_id":"6aa0d0bcd0174964227bece4","name":"Tiantian Mi","hidden":false},{"_id":"6aa0d0bcd0174964227bece5","name":"Pengfei Liu","hidden":false}],"publishedAt":"2026-09-06T00:00:00.000Z","submittedOnDailyAt":"2026-09-09T00:00:00.000Z","title":"VidaForge: Open Research Infrastructure for Video Pretraining Data Recipes","submittedOnDailyBy":{"_id":"633fc70529b5a95f6e15a6b7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/Fzh7wWuqU-fBbzdupOUtF.jpeg","isPro":false,"fullname":"Yan Ma","user":"ManTle","type":"user","name":"ManTle"},"summary":"Video foundation models increasingly rely on large-scale pretraining data, yet the end-to-end data pipelines behind them remain largely closed and difficult to inspect or reuse. Researchers seeking to understand how video data recipes affect model pretraining often need to build substantial infrastructure before testing even a focused hypothesis. We present VIDAFORGE, an open research infrastructure that represents a video data recipe as an executable five-stage workflow from raw videos to training datasets. A decision in this workflow can be varied to construct alternative datasets while preserving how every sample was produced. To demon strate this research workflow, we compare data recipes with different coverage and quality in early from-scratch pretraining of Wan 2.1 and V-JEPA 2.1. Across both learning objectives, the broader-coverage recipe achieves the highest downstream benchmark scores, while loss-based evaluation favors different recipes. This study demonstrates how VidaForge connects data-recipe choices to downstream model performance. We further release VIDAFORGE-3M, containing 3.14 million scene level clips totaling 6,475 hours, with fine-grained annotations and curation signals for video data-recipe research.","upvotes":17,"discussionId":"6aa0d0bdd0174964227bece6","projectPage":"https://github.com/GAIR-NLP/VidaForge","githubRepo":"https://github.com/GAIR-NLP/VidaForge","githubRepoAddedBy":"user","ai_summary":"VIDAFORGE is an open infrastructure that models video data recipes as executable workflows to study how data choices affect pretraining of video foundation models.","ai_keywords":["video foundation models","pretraining","data recipes","VIDAFORGE","executable workflow","Wan 2.1","V-JEPA 2.1","loss-based evaluation","downstream benchmark scores","VIDAFORGE-3M"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":88,"organization":{"_id":"6a5de8695af0052a6ecbf5a1","name":"VidaForge","fullname":"VidaForge","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/hbApm1EAYFONM4h8T-6MT.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"633fc70529b5a95f6e15a6b7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/Fzh7wWuqU-fBbzdupOUtF.jpeg","isPro":false,"fullname":"Yan Ma","user":"ManTle","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"6a6c7b043574d63d5305d6fc","avatarUrl":"/avatars/6e5b4d8ac3a28eee5ad8c18734c2db7b.svg","isPro":false,"fullname":"Patricia Anderson","user":"Rapid-Theo","type":"user"},{"_id":"6a6c7edf33ba949670870a48","avatarUrl":"/avatars/7b5ac17124afb1b3c27315f7307e858c.svg","isPro":false,"fullname":"Michael Jones","user":"SableLens","type":"user"},{"_id":"6a6aa1a9719ef934225fa075","avatarUrl":"/avatars/c7528de1b34e4f12e112800214751369.svg","isPro":false,"fullname":"Matthew Thomas","user":"orbitMatthew","type":"user"},{"_id":"6a8254a78d1455f75eab5b7c","avatarUrl":"/avatars/eeb1e7a5c1aaafd3de1b32e6a61a146f.svg","isPro":false,"fullname":"Minghao Wang","user":"Granite-Minghao","type":"user"},{"_id":"6a9aead6ebd110af46eb1371","avatarUrl":"/avatars/085090a50c11b7c96e8551e5f0956254.svg","isPro":false,"fullname":"张飞","user":"Cobalt-Fox","type":"user"},{"_id":"6a9b56cbfb2bd7a63737a5ae","avatarUrl":"/avatars/94a30df9a1f7dbfeefa9691cf59730ef.svg","isPro":false,"fullname":"高橋さゆり","user":"murakamiatsushi1","type":"user"},{"_id":"6aa0ed5c25e7f6a7f189b227","avatarUrl":"/avatars/a07bb8d1c1b4c6472f2e810945316f71.svg","isPro":false,"fullname":"김영호","user":"sugja07","type":"user"},{"_id":"684d57f26e04c265777ead3f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/cuOj-bQqukSZreXgUJlfm.png","isPro":false,"fullname":"Joakim Lee","user":"Reinforcement4All","type":"user"},{"_id":"6a6aa35203669d40f667ac77","avatarUrl":"/avatars/d1b79b2399d61825f838e48fe252f52a.svg","isPro":false,"fullname":"Karen Rodriguez","user":"Karen-Rodriguez","type":"user"},{"_id":"6a6c8cc8d98e3eb6532ad44d","avatarUrl":"/avatars/782dda92387e078aa8c0e8d95cb417d5.svg","isPro":false,"fullname":"Timothy Rodriguez","user":"Zenith-Eli","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6a5de8695af0052a6ecbf5a1","name":"VidaForge","fullname":"VidaForge","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/hbApm1EAYFONM4h8T-6MT.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2609/2609.06652.md","query":{}}">

Papers

arxiv:2609.06652

VidaForge: Open Research Infrastructure for Video Pretraining Data Recipes

Published on Sep 6

· Submitted by

Yan Ma on Sep 9

VidaForge

Upvote

Authors:

Yan Ma ,

Abstract

VIDAFORGE is an open infrastructure that models video data recipes as executable workflows to study how data choices affect pretraining of video foundation models.

Generated by thinkingmachines/Inkling-Small

Video foundation models increasingly rely on large-scale pretraining data, yet the end-to-end data pipelines behind them remain largely closed and difficult to inspect or reuse. Researchers seeking to understand how video data recipes affect model pretraining often need to build substantial infrastructure before testing even a focused hypothesis. We present VIDAFORGE, an open research infrastructure that represents a video data recipe as an executable five-stage workflow from raw videos to training datasets. A decision in this workflow can be varied to construct alternative datasets while preserving how every sample was produced. To demon strate this research workflow, we compare data recipes with different coverage and quality in early from-scratch pretraining of Wan 2.1 and V-JEPA 2.1. Across both learning objectives, the broader-coverage recipe achieves the highest downstream benchmark scores, while loss-based evaluation favors different recipes. This study demonstrates how VidaForge connects data-recipe choices to downstream model performance. We further release VIDAFORGE-3M, containing 3.14 million scene level clips totaling 6,475 hours, with fine-grained annotations and curation signals for video data-recipe research.