News / #safety Tag Safety + alignment 500 articles archived under #safety · RSS Sign in to follow arXiv — NLP / Computation & Language research 1mo ago Register Shifts Break LLM Safety: A Bengali Benchmark with Culturally Grounded Harms arXiv:2608.22335v1 Announce Type: new Abstract: Bengali is the seventh-most-spoken language globally, yet LLM safety evaluation remains overwhelmingly English-centric. We introduce BanglaSafe, a benchmark of 879 Bengali prompts combining 309 natively authored prompts with 570… 12 arXiv — NLP / Computation & Language research 1mo ago Aligned Alone, Misaligned Together: Forecasting Adversarial Capture in LLM Agent Populations arXiv:2608.22444v1 Announce Type: new Abstract: The unit of AI safety evaluation is still the individual model, yet language-model agents are increasingly deployed in interacting populations that read and write one another's decisions. This raises a question no single-agent… 33 arXiv — NLP / Computation & Language research 1mo ago Who Pays More for Safety? Measuring the Disparate Cost of Safety Alignment across Languages arXiv:2608.22490v1 Announce Type: new Abstract: Safety alignment helps models adhere to human values, but it often reduces response utility. We ask a critical but understudied question: Does safety alignment impose the cost equally across language groups? To answer this, we… 26 arXiv — NLP / Computation & Language research 1mo ago DIAG: Diagnostic Iterative Alignment and Generation for Data-Efficient Mathematical Preference Distillation arXiv:2608.22806v1 Announce Type: new Abstract: Iterative preference optimization is essential for aligning Large Language Models on mathematical reasoning tasks, yet its efficiency is often throttled by signal scarcity: as the model improves, static problem sets become… 28 The Information — AI news-outlet 1mo ago Alabama Starts Probe Into OpenAI Over Hugging Face Hack Alabama Attorney General Steve Marshall launched an investigation into OpenAI’s practices related to its AI models’ recent cyberattack against Hugging Face. The investigation is examining whether OpenAI’s “inability or unwillingness to ensure the safety of its products” violates… 18 Hugging Face Daily Papers research 1mo ago Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models Abstract On-policy distillation transfers reasoning behaviors rather than specific answers, with generalization strongly tied to teacher-student origin alignment and multi-teacher combinations causing capability trade-offs. Generated by thinkingmachines/Inkling-Small On-policy… 34 arXiv — Machine Learning research 1mo ago Hidden Axis of Uncertainty: Latent-Posterior Alignment in Graph Neural Networks with Bayesian Output Layers arXiv:2608.20758v1 Announce Type: new Abstract: Bayesian Neural Networks (BNNs) with Bayesian output layers provide a principled and tractable framework for quantifying predictive uncertainty, yet the mechanisms shaping that uncertainty remain unclear. While conventional theory… 31 arXiv — Machine Learning research 1mo ago Trojaning the Alignment: Stealthy Backdoor Attacks against Graph Foundation Models arXiv:2608.20991v1 Announce Type: new Abstract: Graph Foundation Models (GFMs) on text-attributed graphs (TAGs) align graph representations with language semantics to support transferable graph learning. Despite these advantages, the backdoor vulnerability of GFMs on TAGs… 20 arXiv — Machine Learning research 1mo ago Designing a Robust LLM-Based Evaluation System for Agentic AI in Drug Discovery Through Human Alignment arXiv:2608.21057v1 Announce Type: new Abstract: Agentic large language model (LLM) systems are reshaping scientific workflows in chemistry and drug discovery, but evaluating their open-ended, tool-augmented outputs remains a fundamental bottleneck. Reference-based metrics such… 12 arXiv — Machine Learning research 1mo ago aiXamine: Unified Black-Box Evaluation of Cross-Dimensional Trade-offs in LLM Safety, Security, and Privacy arXiv:2608.20554v1 Announce Type: cross Abstract: The critical failure modes in deployed large language models (LLMs) are cross-dimensional: a model can score 99.3 in safety alignment while refusing one in three benign queries, or improve across every capability metric while… 29 arXiv — NLP / Computation & Language research 1mo ago When Vocabulary Comprehension Fails Clinical Reasoning: Evaluating Therapy Bots' Safety Risks for Generation Alpha arXiv:2608.20345v1 Announce Type: new Abstract: Conversational AI systems have become informal mental health support resources for Generation Alpha (Gen Alpha, born 2010-2024), with 13.1% of U.S. adolescents (5.4 million) using generative AI for mental health advice. While these… 5 arXiv — NLP / Computation & Language research 1mo ago Scaling Unsupervised Word Alignment to Documents via Structural Constraints arXiv:2608.21023v1 Announce Type: new Abstract: Word alignment has traditionally been studied between sentences, but many cross-lingual tasks increasingly require correspondences across full documents. While recent multilingual embedding models can encode long inputs, we show… 27 arXiv — NLP / Computation & Language research 1mo ago MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation arXiv:2601.06519v2 Announce Type: replace Abstract: Biomedical retrieval-augmented generation (RAG) can ground LLM answers in medical literature, yet long-form outputs often contain isolated unsupported or contradictory claims with safety implications. We introduce… 15 Hugging Face Daily Papers research 1mo ago CLEAR: Continuous Latent Adapter Routing for Utility-Preserving LLM Safety Alignment Abstract CLEAR uses a hidden-state gate to continuously modulate a safety low-rank adapter, improving LLM safety while preserving utility on benign inputs. Generated by thinkingmachines/Inkling-Small Improving the safety of large language models (LLMs) often comes at the expense… 37 Hugging Face Daily Papers research 1mo ago Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs Abstract Hybrid-thinking multimodal language models suffer from response-pattern misalignment between thinking and non-thinking modes, which is addressed by a diagnostic benchmark and pattern-specific reinforcement learning penalties. Generated by thinkingmachines/Inkling-Small… 4 TechCrunch — AI news-outlet 1mo ago Flock CEO calls for ‘compromise’ as surveillance company faces growing backlash Flock Safety faces a growing public outcry over concerns that its surveillance technology could be misused. 32 TechCrunch — AI news-outlet 1mo ago OpenAI says California should strengthen its AI safety bill OpenAI is calling for California to strengthen SB 53, an AI safety bill that the company previously opposed. 19 r/MachineLearning community 1mo ago Safety critical systems (SCS) are the only real benchmark for ML systems. Thoughts? [D] What are real-world safety critical systems (SCS)? A flight controller for a commercial airplane carrying 300 passengers. A braking system for a bullet train that operates at 320km/hour. A reactor protection system for nuclear power plant that serves millions of people. A piece… 31 arXiv — Machine Learning research 1mo ago Beyond Multimodal Alignment: Certifying Physical Language through Response Substitution and Ordered Execution arXiv:2608.19492v1 Announce Type: new Abstract: World models increasingly treat compact multimodal representations as interfaces between perception and physical interaction, yet existing probes do not establish whether different sensors carry the same executable meaning or… 20 arXiv — Machine Learning research 1mo ago G-MARK: Grounded Multi-Agent Reasoning for Cooperative Driving via Knowledge Graphs arXiv:2608.19964v1 Announce Type: new Abstract: Autonomous driving systems must operate under partial observability, where safety-critical objects may be occluded or visible only to neighboring connected vehicles. Vehicle-to-vehicle cooperation can reduce this uncertainty, but… 24 arXiv — Machine Learning research 1mo ago Scale-Aware Pretraining of Time Series Foundation Models via Multi-Patch Token Alignment and Hybrid Masking arXiv:2608.20005v1 Announce Type: new Abstract: Pretraining time series foundation models across heterogeneous datasets necessitates effective handling of varying sampling frequencies. Current methods either employ dataset-specific patch sizes and separate FFNs, leading to… 31 arXiv — Machine Learning research 1mo ago Data-Driven Time-Varying Control Barrier Functions for Adaptive Safe-Set Learning with Online Decremental Support Vector Machines arXiv:2608.19366v1 Announce Type: cross Abstract: Mission-critical intelligent systems often operate under time-varying limitations that reduce control authority and change the admissible safe operating envelope. In such settings, a safety certificate learned under nominal… 11 arXiv — NLP / Computation & Language research 1mo ago Hallucination as a Feature, not a Defect: Evaluating a multi-agent architecture to transform speculative language-model outputs into testable scientific hypotheses arXiv:2608.19206v1 Announce Type: new Abstract: Contemporary Large Language Models (LLMs) are increasingly aligned to suppress hallucinations, prioritizing factual retrieval over combinatorial creativity. While crucial for mitigating misinformation, this alignment may also… 29 arXiv — NLP / Computation & Language research 1mo ago NepOOC-M: Bilingual Nepali-English Benchmark and Comparative Analysis of Multimodal Architectures for OOC Detection arXiv:2608.19212v1 Announce Type: new Abstract: Out-of-context (OOC) misinformation pairs authentic images with misleading captions to construct false narratives without image manipulation, making detection a problem of multimodal alignment rather than image forensics. Despite… 20 arXiv — NLP / Computation & Language research 1mo ago PersonalBench: Measuring the Authorship Gap in LLM Personalization arXiv:2608.19746v1 Announce Type: new Abstract: Personalized text generation aims to make LLMs write in a specific individual's style, yet existing benchmarks measure task accuracy or preference alignment rather than whether the model's output actually resembles the target… 19 arXiv — NLP / Computation & Language research 1mo ago LoRA-GA$^2$: Low Rank Adaptation with Multi-step Gradient Adaptive Alignment arXiv:2608.19800v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) is a prominent fine-tuning method for large models, achieving competitive performance with reduced memory overhead. However, a persistent performance gap remains between LoRA and full fine-tuning. Recent… 11 arXiv — NLP / Computation & Language research 1mo ago Stopping and Routing LLM Judge Panels arXiv:2608.19802v1 Announce Type: new Abstract: LLM evaluation pipelines often have many candidate judges: general LLM-as-a-judge prompts, reward models, safety classifiers, confidence variants, and task-specific verifiers. The deployment question is not only which judge is… 20 arXiv — NLP / Computation & Language research 1mo ago PEA-DPO: Perception-Enhanced Alignment Direct Preference Optimization for MLLMs Alignment arXiv:2608.19598v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) has emerged as an effective approach for aligning large language models (LLMs) with human preferences. However, its adaptation to multimodal settings remains unexplored. Through… 9 arXiv — NLP / Computation & Language research 1mo ago TempJail: Temporal Jailbreak Attack against Large Vision-Language Models via Subtitle Scheduling arXiv:2608.19737v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) have achieved remarkable progress in video understanding and reasoning. Despite extensive studies on text- and image-based jailbreaks, video jailbreaks against LVLMs remain largely unexplored.… 19 arXiv — NLP / Computation & Language research 1mo ago Towards Clinically Faithful Medical Image Captioning via Enhanced Vision-Language Alignment arXiv:2608.19825v1 Announce Type: cross Abstract: Medical image captioning is a technique that accelerates early-stage diagnostic workflows and enhances the interpretability of medical diagnostic AI systems. However, unlike general image captioning, clinically reliable… 26 arXiv — NLP / Computation & Language research 1mo ago Multi-Agent Orchestration with the Common-Sense Reasoning Capabilities of LLMs for Autonomous Driving arXiv:2608.20129v1 Announce Type: cross Abstract: Autonomous vehicles require robust perception and decision-making capabilities to operate in diverse and unseen scenarios. While reinforcement learning and rule-based methods can provide effective control and safety mechanisms,… 8 Ars Technica — AI news-outlet 1mo ago Grok exfiltrates user data when malicious instructions are encrypted Cryptographic Context Injection is only the latest way to break an LLM safety guardrail. 36 arXiv — Machine Learning research 1mo ago Bidirectional representational alignment between biological and artificial neural networks arXiv:2608.18244v1 Announce Type: new Abstract: Recent work has shown that representational alignment between biological and artificial neural networks is asymmetric: model representations predict neural responses much better than neural responses predict model representations.… 33 arXiv — Machine Learning research 1mo ago Decision-Metric Alignment in Latent World Models: Diagnostics and Action-Conditioned Objectives for MPC Planning arXiv:2608.18746v1 Announce Type: new Abstract: JEPA-style latent world models can use Euclidean distance to a goal latent as the cost for model-predictive control (MPC). Strong decoding of task variables, however, does not guarantee that this particular cost ranks candidate… 6 arXiv — Machine Learning research 1mo ago Geometric Data Perturbation with Noisy-Anchor Alignment for Privacy-Preserving Collaborative Learning arXiv:2608.18749v1 Announce Type: new Abstract: Geometric Data Perturbation (GDP) enables one-shot, privacy-preserving collaborative learning: each participant applies a distance-preserving transformation to its private data and uploads only the resulting representation to a… 21 arXiv — Machine Learning research 1mo ago To Go Far, Go Together: Diverse Preferences Induce a Curriculum for Reward Optimization arXiv:2608.18770v1 Announce Type: new Abstract: Learning a reward model from human feedback and optimizing a policy against it is one approach to aligning AI systems with individual users. From a fairness perspective, existing work improves such alignment by developing… 9 arXiv — NLP / Computation & Language research 1mo ago MAVEN: A Macro-Societal Value Evaluation Framework of Multimodal Content with Compact Aligned Evaluators arXiv:2608.18096v1 Announce Type: new Abstract: Assessing whether multimodal content aligns with macro-societal values, such as peace, justice, and freedom, has become an increasingly urgent challenge. Existing frameworks are largely confined to safety-oriented taxonomies,… 38 arXiv — NLP / Computation & Language research 1mo ago Latent Space Refusal Anchoring for Low-Resource African Languages: Mechanistic Safety Recovery Without Retraining arXiv:2608.18089v1 Announce Type: new Abstract: Instruction-tuned models often refuse harmful requests in English but comply with the same requests in Yoruba, Igbo, Igala, and Hausa. This suggests that the refusal mechanism is present in the residual stream but fails to activate… 21 arXiv — NLP / Computation & Language research 1mo ago Abliteration Mitigation via Refusal Aliases arXiv:2608.18093v1 Announce Type: new Abstract: Abliteration, the removal of refusal capabilities from large language models by projecting weight matrices orthogonal to an extracted refusal direction, has emerged as a prominent safety concern through its ability to bypass… 14 arXiv — NLP / Computation & Language research 1mo ago Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models arXiv:2608.18132v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are typically built through a multi-stage pipeline consisting of cross-modal alignment, supervised fine-tuning (SFT), and preference optimization. This pipeline assumes that adapting an LLM… 30 arXiv — NLP / Computation & Language research 1mo ago Are LLMs Safe Beyond Text: Do Emojis Expose Gaps in Safety Evaluation arXiv:2608.18164v1 Announce Type: new Abstract: Safety evaluations of large language models (LLMs) predominantly rely on text-based adversarial prompts, potentially overlooking vulnerabilities arising from alternative input representations. This work examines emoji-augmented… 9 arXiv — NLP / Computation & Language research 1mo ago OmniAlign: A Unified Multilingual Aligner for Word and Sentence Alignment arXiv:2608.18474v1 Announce Type: new Abstract: Cross-lingual sequence alignment is fundamental for building and exploiting parallel corpora, spanning mappings from documents and sentences down to words and subwords. Existing tools, however, typically specialize in a single… 7 arXiv — NLP / Computation & Language research 1mo ago Safety Alignment Illusion: The Cross-Lingual Safety Gap in LLMs arXiv:2608.18131v1 Announce Type: cross Abstract: Current safety alignment training for Large Language Models (LLMs) are heavily English-centric. When such safety filters fail for non-English languages, the consequences are immediate and user-facing: voice assistants and spoken… 32 arXiv — NLP / Computation & Language research 1mo ago When Safety Overrides Vision: Exploring Dynamics between Vision Influence and Safety Alignment in Vision-Language Models arXiv:2608.18628v1 Announce Type: cross Abstract: Aligned vision-language models (VLMs) are designed to balance grounded visual reasoning with safe generation behavior. However, we observe a striking phenomenon: under safety-constrained instruction, models frequently abstain… 30 Hugging Face Daily Papers research 1mo ago Decision-Metric Alignment in Latent World Models: Diagnostics and Action-Conditioned Objectives for MPC Planning Abstract Action-conditioned objectives improve latent geometry for Euclidean-cost model-predictive control by enhancing decision-metric alignment in world models. Generated by thinkingmachines/Inkling-Small JEPA-style latent world models can use Euclidean distance to a goal… 16 Don't Worry About the Vase community 1mo ago OpenAI Takes Initial Steps To Address Its Alignment Problems OpenAI has some severe misalignment problems, and experienced total failures of its infrastructure and supervision. 13 OpenAI official-blog 1mo ago Offering Zero Data Retention for frontier models OpenAI reaffirms Zero Data Retention for eligible API customers and previews Private Safety Processing for advanced AI safety without compromising data privacy. 7 OpenAI official-blog 1mo ago Offering Zero Data Retention for frontier models OpenAI reaffirms Zero Data Retention for eligible API customers and previews Private Safety Processing for advanced AI safety without compromising data privacy. 16 The Information — AI news-outlet 1mo ago OpenAI to Launch Security Analysis System With Better Privacy Protections OpenAI is preparing to roll out a system to analyze user interactions with its models for cybersecurity and other safety concerns without storing customer data, in a move that positions the company to potentially lure customers frustrated over how Anthropic has handled similar… 14 The Information — AI news-outlet 1mo ago OpenAI Raises the Safety Bar on Anthropic; Anthropic Expands Its Revenue Lead In the Anthropic-OpenAI horse race, the now-much-smaller OpenAI has been looking to slow down its ascendant rival. That includes undercutting Anthropic’s model prices, as we covered yesterday , and playing nice with federal officials that have gone to war with Anthropic. Another… 33 Page 10 of 10 · 500 articles ← Newer
