Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior. Existing safeguards, however, are typically trained for a single judgment target and reduce safety assessment to a binary decision. Consequently, risk becomes difficult to compare across a multimodal interaction, and ambiguous cases are obscured. We introduce SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale. We curate a broad collection of safety-relevant data from both real-world and synthetic sources and apply a disagreement-aware annotation procedure. The resulting dataset spans 15 harm categories and 55 fine-grained subcategories, covering a broad range of multimodal safety scenarios. We also construct SafeAtlas-Bench, a held-out set of 5,000 instances for evaluating five-level predictions and continuous risk scores. Upon this dataset, we train the SafeAtlas Guard series of models via target-conditioned tuning for multimodal safety detection. Our models not only perform five-way classification of safety levels but also map safety to continuous scores through a soft cumulative ordinal head. Experimental results demonstrate that guard models trained on our dataset exhibit strong generalization: even without using the training sets of other benchmarks, they achieve competitive performance on the corresponding test sets. Notably, our 8B model attains the overall best performance, outperforming the previous SOTA by approximately 4% in F1 score. Code, data, and models are released to support further research. Warning: this paper contains example data that may be offensive, harmful, graphic, or disturbing.</p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/69bb034a920c63f808fe2184/stbUFTohFbLsVk1WP9OPu.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/69bb034a920c63f808fe2184/stbUFTohFbLsVk1WP9OPu.png\" alt=\"overview\"></a></p>\n","updatedAt":"2026-09-01T08:27:55.631Z","author":{"_id":"69bb034a920c63f808fe2184","avatarUrl":"/avatars/c201052486a96624e53a81cffa3032dd.svg","fullname":"Zongrui Wang","name":"zrwang1211","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8999638557434082},"editors":["zrwang1211"],"editorAvatarUrls":["/avatars/c201052486a96624e53a81cffa3032dd.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.29098","authors":[{"_id":"6a96619acd6ebc484732ee32","user":{"_id":"69bb034a920c63f808fe2184","avatarUrl":"/avatars/c201052486a96624e53a81cffa3032dd.svg","isPro":false,"fullname":"Zongrui Wang","user":"zrwang1211","type":"user","name":"zrwang1211"},"name":"Zongrui Wang","status":"claimed_verified","statusLastChangedAt":"2026-09-01T08:08:22.466Z","hidden":false},{"_id":"6a96619acd6ebc484732ee33","name":"Xiangyang Zhu","hidden":false},{"_id":"6a96619acd6ebc484732ee34","name":"Sicheng Wang","hidden":false},{"_id":"6a96619acd6ebc484732ee35","name":"Han Wang","hidden":false},{"_id":"6a96619acd6ebc484732ee36","name":"Dingyi Rong","hidden":false},{"_id":"6a96619acd6ebc484732ee37","name":"Zeyu Zhang","hidden":false},{"_id":"6a96619acd6ebc484732ee38","name":"Chunyi Li","hidden":false},{"_id":"6a96619acd6ebc484732ee39","name":"Yue Shi","hidden":false},{"_id":"6a96619acd6ebc484732ee3a","name":"Kaiwei Zhang","hidden":false},{"_id":"6a96619acd6ebc484732ee3b","name":"Zicheng Zhang","hidden":false},{"_id":"6a96619acd6ebc484732ee3c","name":"Yuan Tian","hidden":false},{"_id":"6a96619acd6ebc484732ee3d","name":"Qi Jia","hidden":false},{"_id":"6a96619acd6ebc484732ee3e","name":"Yan Teng","hidden":false},{"_id":"6a96619acd6ebc484732ee3f","name":"Wei Sun","hidden":false},{"_id":"6a96619acd6ebc484732ee40","name":"Ning Liu","hidden":false},{"_id":"6a96619acd6ebc484732ee41","name":"Guangtao Zhai","hidden":false}],"publishedAt":"2026-08-29T00:00:00.000Z","submittedOnDailyAt":"2026-09-01T00:00:00.000Z","title":"SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models","submittedOnDailyBy":{"_id":"69bb034a920c63f808fe2184","avatarUrl":"/avatars/c201052486a96624e53a81cffa3032dd.svg","isPro":false,"fullname":"Zongrui Wang","user":"zrwang1211","type":"user","name":"zrwang1211"},"summary":"Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior. Existing safeguards, however, are typically trained for a single judgment target and reduce safety assessment to a binary decision. Consequently, risk becomes difficult to compare across a multimodal interaction, and ambiguous cases are obscured. We introduce SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale. We curate a broad collection of safety-relevant data from both real-world and synthetic sources and apply a disagreement-aware annotation procedure. The resulting dataset spans 15 harm categories and 55 fine-grained subcategories, covering a broad range of multimodal safety scenarios. We also construct SafeAtlas-Bench, a held-out set of 5,000 instances for evaluating five-level predictions and continuous risk scores. Upon this dataset, we train the SafeAtlas Guard series of models via target-conditioned tuning for multimodal safety detection. Our models not only perform five-way classification of safety levels but also map safety to continuous scores through a soft cumulative ordinal head. Experimental results demonstrate that guard models trained on our dataset exhibit strong generalization: even without using the training sets of other benchmarks, they achieve competitive performance on the corresponding test sets. Notably, our 8B model attains the overall best performance, outperforming the previous SOTA by approximately 4% in F1 score. Code, data, and models are released to support further research. Warning: this paper contains example data that may be offensive, harmful, graphic, or disturbing.","upvotes":2,"discussionId":"6a96619bcd6ebc484732ee42","githubRepo":"https://github.com/zrwang1211/SafeAtlas-VL","githubRepoAddedBy":"user","ai_summary":"SafeAtlas-VL introduces a large-scale multimodal safety dataset with graded risk labels and trains guard models that predict continuous safety scores, achieving state-of-the-art generalization.","ai_keywords":["multimodal safety moderation","five-level ordered scale","disagreement-aware annotation","target-conditioned tuning","soft cumulative ordinal head","SafeAtlas Guard"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":3,"organization":{"_id":"63e5ef7bf2e9a8f22c515654","name":"SJTU","fullname":"Shanghai Jiao Tong University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1676013394657-63e5ee22b6a40bf941da0928.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"69bb034a920c63f808fe2184","avatarUrl":"/avatars/c201052486a96624e53a81cffa3032dd.svg","isPro":false,"fullname":"Zongrui Wang","user":"zrwang1211","type":"user"},{"_id":"63b3ebc7b7fec0adf64e3c8a","avatarUrl":"/avatars/2eb0ce27db0b83dd488bbc1ad3d45a1d.svg","isPro":false,"fullname":"Zhu Xiangyang","user":"yyy127","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"63e5ef7bf2e9a8f22c515654","name":"SJTU","fullname":"Shanghai Jiao Tong University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1676013394657-63e5ee22b6a40bf941da0928.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.29098.md","query":{}}">
SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models
Abstract
SafeAtlas-VL introduces a large-scale multimodal safety dataset with graded risk labels and trains guard models that predict continuous safety scores, achieving state-of-the-art generalization.
Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior. Existing safeguards, however, are typically trained for a single judgment target and reduce safety assessment to a binary decision. Consequently, risk becomes difficult to compare across a multimodal interaction, and ambiguous cases are obscured. We introduce SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale. We curate a broad collection of safety-relevant data from both real-world and synthetic sources and apply a disagreement-aware annotation procedure. The resulting dataset spans 15 harm categories and 55 fine-grained subcategories, covering a broad range of multimodal safety scenarios. We also construct SafeAtlas-Bench, a held-out set of 5,000 instances for evaluating five-level predictions and continuous risk scores. Upon this dataset, we train the SafeAtlas Guard series of models via target-conditioned tuning for multimodal safety detection. Our models not only perform five-way classification of safety levels but also map safety to continuous scores through a soft cumulative ordinal head. Experimental results demonstrate that guard models trained on our dataset exhibit strong generalization: even without using the training sets of other benchmarks, they achieve competitive performance on the corresponding test sets. Notably, our 8B model attains the overall best performance, outperforming the previous SOTA by approximately 4% in F1 score. Code, data, and models are released to support further research. Warning: this paper contains example data that may be offensive, harmful, graphic, or disturbing.
Community
Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior. Existing safeguards, however, are typically trained for a single judgment target and reduce safety assessment to a binary decision. Consequently, risk becomes difficult to compare across a multimodal interaction, and ambiguous cases are obscured. We introduce SafeAtlas-VL, a dataset of 1.5M training instances that places image-, request-, and response-level judgments on a five-level ordered scale. We curate a broad collection of safety-relevant data from both real-world and synthetic sources and apply a disagreement-aware annotation procedure. The resulting dataset spans 15 harm categories and 55 fine-grained subcategories, covering a broad range of multimodal safety scenarios. We also construct SafeAtlas-Bench, a held-out set of 5,000 instances for evaluating five-level predictions and continuous risk scores. Upon this dataset, we train the SafeAtlas Guard series of models via target-conditioned tuning for multimodal safety detection. Our models not only perform five-way classification of safety levels but also map safety to continuous scores through a soft cumulative ordinal head. Experimental results demonstrate that guard models trained on our dataset exhibit strong generalization: even without using the training sets of other benchmarks, they achieve competitive performance on the corresponding test sets. Notably, our 8B model attains the overall best performance, outperforming the previous SOTA by approximately 4% in F1 score. Code, data, and models are released to support further research. Warning: this paper contains example data that may be offensive, harmful, graphic, or disturbing.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.29098 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.