News / #safety Tag Safety + alignment 500 articles archived under #safety · RSS Sign in to follow Ars Technica — AI news-outlet 1d ago Meta makes AI glasses slightly less creepy with limit on nonconsensual recording Meta fixes AI glasses to stop recording any time users cover up the safety light. 19 Hugging Face Daily Papers research 1d ago Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher Abstract Self-OPD eliminates task-specific teachers in flow matching by using self-explored stochastic branches and normalized advantages to optimize the velocity field for multi-objective alignment. Generated by thinkingmachines/Inkling-Small On-policy distillation (OPD), which… 26 arXiv — Machine Learning research 2d ago NeuronFuzz: Safety Neuron Guided Fuzzing for LLM Safety Evaluation arXiv:2608.26222v1 Announce Type: new Abstract: Safety evaluation is critical for assessing whether aligned Large Language Models (LLMs) remain robust against jailbreak attacks. Existing automated testing methods, however, largely rely on response-level feedback: each candidate… 12 arXiv — Machine Learning research 2d ago Privacy Without Regret: Differentially Private Inference-Time Alignment arXiv:2608.26324v1 Announce Type: new Abstract: Best-of-N (BoN) sampling is the simplest and most widely deployed inference-time alignment strategy, but it suffers from two distinct problems: reward hacking, in which the selected response exploits errors in the proxy reward… 14 arXiv — Machine Learning research 2d ago A Single Suffix to Break Them All: Basin-Aware Jailbreaks for Merged Model Families arXiv:2608.26506v1 Announce Type: new Abstract: Model merging enables combining multiple fine-tuned models without additional training, but its safety implications remain poorly understood. Prior work primarily attributes merging risks to unsafe constituent models, implicitly… 24 arXiv — Machine Learning research 2d ago GRAS: Guided Reduced-Variance Proposals and Adaptive Selection for Training-Free Reward Alignment in Discrete Diffusion arXiv:2608.26585v1 Announce Type: new Abstract: Discrete diffusion models have become a strong, widely adopted class of generators for sequence data, and steering them toward a downstream reward at inference time, without any retraining, is increasingly important. Such… 24 arXiv — Machine Learning research 2d ago Safety by Design: Realized-Cost Constraints for Contextual Bandits with Continuous Actions arXiv:2608.26755v1 Announce Type: new Abstract: Contextual bandits are a standard framework for sequential decision-making under uncertainty, with applications in clinical trials, dosage selection, recommendation systems, and autonomous systems. Safety is central in many of… 18 arXiv — Machine Learning research 2d ago Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Traffic arXiv:2608.26860v1 Announce Type: new Abstract: Connected and automated vehicle (CAV) platooning offers a promising approach to improving road safety and traffic capacity. However, platoon control in real-world traffic is challenging due to uncertainty and heterogeneous driving… 15 arXiv — NLP / Computation & Language research 2d ago DeflectBench: A Benchmark for Evaluating Rhetorical Fallacy Generation in LLMs arXiv:2608.26119v1 Announce Type: new Abstract: Whether large language models can be prompted to generate rhetorical fallacies on demand, and whether current safety post-training constrains this behavior, has received less attention than the related question of detecting… 30 arXiv — NLP / Computation & Language research 2d ago SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning arXiv:2608.26550v1 Announce Type: new Abstract: Reinforcement learning-based knowledge distillation has the potential to transfer complex reasoning from teacher to student models, yet it currently faces a critical dilemma: researchers must choose between sparse outcome-based… 34 arXiv — NLP / Computation & Language research 2d ago Surgical Alignment in Knowledge Graph Training for Clinical Diagnosis with Large Language Models arXiv:2608.26587v1 Announce Type: new Abstract: Biomedical knowledge graphs (KGs) offer structured medical knowledge that can ground large language model (LLM) reasoning in clinical diagnosis application, yet how KG signal should be integrated into LLMs remains an open question.… 8 arXiv — NLP / Computation & Language research 2d ago ITL: Interpretable Document Alignment with Structured Reference Frameworks arXiv:2608.27031v1 Announce Type: new Abstract: Measuring alignment between documents and structured reference frameworks requires identifying conceptual evidence distributed throughout the text and reporting it through measures that are quantitative, interpretable, and… 19 arXiv — NLP / Computation & Language research 2d ago Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations? arXiv:2608.27115v1 Announce Type: new Abstract: Cross-lingual alignment in multilingual language models is typically attributed to joint training: shared parameters, mixed-language batches, or explicit alignment objectives. We ask whether monolingual models trained on… 8 arXiv — NLP / Computation & Language research 2d ago STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translation arXiv:2608.27161v1 Announce Type: new Abstract: Large Language Models (LLMs) have enabled a shift from sentence-level to document-to-document (Doc2Doc) machine translation, promising improved global coherence. However, document-to-document generation in a single pass frequently… 11 Hugging Face Daily Papers research 2d ago PAWBench: How Far Are We from Probabilistically Aligned World Modeling? Abstract The study formalizes probabilistic alignment for world models, introduces PAWBench and PAWEval to evaluate video generators as stochastic samplers, and finds current models fail to match reference behavior distributions. Generated by thinkingmachines/Inkling-Small… 8 Hugging Face Daily Papers research 3d ago FIRM-Video: Check Before You Score for Reliable Text-to-Video Reward Modeling Abstract FIRM-Video uses checklist-driven verification of temporal visual evidence to build reliable reward models for text-to-video evaluation and alignment. Generated by thinkingmachines/Inkling-Small Reliable reward models are essential for text-to-video evaluation and… 32 arXiv — Machine Learning research 3d ago Multi-Modal Anomaly Detection: A Survey arXiv:2608.24937v1 Announce Type: new Abstract: Multi-Modal Anomaly Detection (MMAD) detects rare abnormal events from heterogeneous data sources and is increasingly used in safety- and reliability-critical applications such as industrial inspection and cybersecurity. Yet the… 9 arXiv — NLP / Computation & Language research 3d ago Learning Mixtures of Plackett-Luce Models for Multi-Objective Alignment arXiv:2608.25200v1 Announce Type: cross Abstract: We consider the problem of learning a mixture of $k$ Plackett-Luce models given multi-way ranking responses from annotators that may represent heterogeneous underlying preferences. This problem has many applications in AI… 30 arXiv — Machine Learning research 3d ago Refusal geometry reflects refusal training: diverse refusal prefixes can raise stable rank and weaken refusal vector ablation attacks arXiv:2608.25390v1 Announce Type: new Abstract: Refusal training protects AI models from jailbreaks by training models to decline unsafe queries, reducing the risk of misuse. Recent work finds that refusal behavior in aligned language models can be mediated by a single… 32 arXiv — NLP / Computation & Language research 3d ago Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal arXiv:2608.24988v1 Announce Type: new Abstract: Activation steering can be embedded directly into a language model's weights, shaping behaviour without inference-time intervention and offering a way to encode alignment prior to release. However, models are routinely fine-tuned… 28 arXiv — NLP / Computation & Language research 3d ago Controllable Affective Generation via Latent Vector Steering arXiv:2608.25569v1 Announce Type: new Abstract: Large Language Models (LLMs) often produce emotionally flattened responses after alignment, limiting their effectiveness in affect-sensitive applications. In this paper, we propose EmoVec, a lightweight framework for controllable… 20 arXiv — NLP / Computation & Language research 3d ago Localize-Then-Decide Guarantees for LLM Judgments arXiv:2608.25824v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as evaluators to assess output quality and preference alignment, yet providing reliable guarantees of agreement with human judgments remains challenging. Recent work introduces… 21 arXiv — NLP / Computation & Language research 3d ago RefLAM: A Reference-Grounded Line Annotation Pipeline for Historical Arabic Manuscripts arXiv:2608.25140v1 Announce Type: cross Abstract: Existing approaches to building line-level Arabic handwritten-text-recognition (HTR) training data either rely on fully manual annotation, which does not scale, or on automatic OCR-to-reference alignment methods not yet extended… 33 arXiv — NLP / Computation & Language research 3d ago Formal, Executable and Explainable Runtime Monitoring of Spoken Air Traffic Control Operational Procedures arXiv:2608.25926v1 Announce Type: cross Abstract: Air traffic control procedures are executed through spoken exchanges between controllers and pilots. These interactions are essential to the safety of air transportation: failures in their execution can create severe operational… 21 Hugging Face Daily Papers research 3d ago When "Must" Becomes "Maybe": Constraint Weakening in LLM Agent Workflows Abstract Multi-stage LLM workflows lose operational constraints when intermediate artifacts transform binding prerequisites into non-binding context, causing safety failures despite preserved content. Generated by thinkingmachines/Inkling-Small Large language model (LLM) agents… 32 Hugging Face Daily Papers research 3d ago Automata from Agent Traces: Failure and Next-Step Prediction Abstract LLM agent traces are compressed into compact finite-state machines that enable accurate next-step and failure prediction for safety auditing and runtime monitoring. Generated by thinkingmachines/Inkling-Small LLM-based agents execute multi-step tasks, but their… 23 arXiv — Machine Learning research 4d ago Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Edge-Deployable Virtual Sensing arXiv:2608.23987v1 Announce Type: new Abstract: Virtual sensing enables digital twins and safety-critical systems to reconstruct and forecast spatial-temporal physics in real time. However, conventional computational and data-driven methods often face challenges in… 14 arXiv — Machine Learning research 4d ago Joint Distribution Alignment for Universal Domain Adaptation arXiv:2608.24429v1 Announce Type: new Abstract: Unsupervised domain adaptation (UDA) has been widely concerned in the fields of machine learning, pattern recognition, and computer vision. Traditional UDA learning usually assumes that the label spaces of the source and target… 17 arXiv — Machine Learning research 4d ago $(\text{DNN})^2$: Doubly Non-Negative Relaxations for Deep Neural Networks arXiv:2608.24743v1 Announce Type: new Abstract: Existing linear program (LP) and semidefinite program (SDP) relaxations for rectified linear unit (ReLU) neural network (NN) verification yield overly-conservative safety guarantees due to significant relaxation gaps. While the… 6 arXiv — Machine Learning research 4d ago Automata from Agent Traces: Failure and Next-Step Prediction arXiv:2608.23670v1 Announce Type: cross Abstract: LLM-based agents execute multi-step tasks, but their behavioral structure remains opaque: long unstructured traces resist the safety auditing and runtime monitoring that deployment requires. Existing approaches operate per-trace… 5 Hugging Face Daily Papers research 4d ago On-Policy Self-Distillation in Diffusion Models Abstract DiffusionOPSD uses on-policy self-distillation to turn image-level rewards into explicit intermediate targets for diffusion models, improving alignment efficiency and enabling separate analysis of target construction and policy fitting. Generated by… 16 OpenAI official-blog 4d ago The Hugging Face incident and the road ahead OpenAI shares findings from the Hugging Face security incident and the steps we’re taking to strengthen AI model security, monitoring, and alignment. 21 Hugging Face Daily Papers research 4d ago GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture Abstract GigaBrain-0.7 is a vision-language-action model that improves embodied generalization via a three-system architecture, large-scale heterogeneous pretraining, and joint alignment training. Generated by thinkingmachines/Inkling-Small Vision-language-action (VLA) models… 31 arXiv — NLP / Computation & Language research 5d ago Evaluation Awareness in Language Models: Representation, Verbalization, and Control arXiv:2608.21766v1 Announce Type: new Abstract: Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their behavior in deployment. This assumption can fail, should models infer that they are… 16 arXiv — NLP / Computation & Language research 5d ago No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios arXiv:2608.21775v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in real-world applications, yet they remain vulnerable to generating harmful content. From adversarial jailbreaks that bypass safety filters to implicit hate that evades… 30 arXiv — NLP / Computation & Language research 5d ago BanglaVeilGuard: Cross-Script Safety Benchmarking and Lightweight Guardrails for Bangla Large Language Models arXiv:2608.21880v1 Announce Type: new Abstract: Bangla large language model (LLM) safety is difficult to evaluate with English-centric or standard-script benchmarks because Bangla users routinely write across scripts, spellings, code-mixed forms, and regional registers. This… 4 arXiv — NLP / Computation & Language research 5d ago Text-Anchored Semantic Perturbations for Transferable Jailbreak Attacks on Multimodal Large Language Models arXiv:2608.22312v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language interaction, yet their safety alignment remains vulnerable to jailbreak attacks. A key challenge is that safety behavior learned in the… 13 arXiv — NLP / Computation & Language research 5d ago Register Shifts Break LLM Safety: A Bengali Benchmark with Culturally Grounded Harms arXiv:2608.22335v1 Announce Type: new Abstract: Bengali is the seventh-most-spoken language globally, yet LLM safety evaluation remains overwhelmingly English-centric. We introduce BanglaSafe, a benchmark of 879 Bengali prompts combining 309 natively authored prompts with 570… 12 arXiv — NLP / Computation & Language research 5d ago Aligned Alone, Misaligned Together: Forecasting Adversarial Capture in LLM Agent Populations arXiv:2608.22444v1 Announce Type: new Abstract: The unit of AI safety evaluation is still the individual model, yet language-model agents are increasingly deployed in interacting populations that read and write one another's decisions. This raises a question no single-agent… 33 arXiv — NLP / Computation & Language research 5d ago Who Pays More for Safety? Measuring the Disparate Cost of Safety Alignment across Languages arXiv:2608.22490v1 Announce Type: new Abstract: Safety alignment helps models adhere to human values, but it often reduces response utility. We ask a critical but understudied question: Does safety alignment impose the cost equally across language groups? To answer this, we… 26 arXiv — NLP / Computation & Language research 5d ago DIAG: Diagnostic Iterative Alignment and Generation for Data-Efficient Mathematical Preference Distillation arXiv:2608.22806v1 Announce Type: new Abstract: Iterative preference optimization is essential for aligning Large Language Models on mathematical reasoning tasks, yet its efficiency is often throttled by signal scarcity: as the model improves, static problem sets become… 28 The Information — AI news-outlet 5d ago Alabama Starts Probe Into OpenAI Over Hugging Face Hack Alabama Attorney General Steve Marshall launched an investigation into OpenAI’s practices related to its AI models’ recent cyberattack against Hugging Face. The investigation is examining whether OpenAI’s “inability or unwillingness to ensure the safety of its products” violates… 18 Hugging Face Daily Papers research 6d ago Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models Abstract On-policy distillation transfers reasoning behaviors rather than specific answers, with generalization strongly tied to teacher-student origin alignment and multi-teacher combinations causing capability trade-offs. Generated by thinkingmachines/Inkling-Small On-policy… 34 arXiv — Machine Learning research 6d ago Hidden Axis of Uncertainty: Latent-Posterior Alignment in Graph Neural Networks with Bayesian Output Layers arXiv:2608.20758v1 Announce Type: new Abstract: Bayesian Neural Networks (BNNs) with Bayesian output layers provide a principled and tractable framework for quantifying predictive uncertainty, yet the mechanisms shaping that uncertainty remain unclear. While conventional theory… 31 arXiv — Machine Learning research 6d ago Trojaning the Alignment: Stealthy Backdoor Attacks against Graph Foundation Models arXiv:2608.20991v1 Announce Type: new Abstract: Graph Foundation Models (GFMs) on text-attributed graphs (TAGs) align graph representations with language semantics to support transferable graph learning. Despite these advantages, the backdoor vulnerability of GFMs on TAGs… 20 arXiv — Machine Learning research 6d ago Designing a Robust LLM-Based Evaluation System for Agentic AI in Drug Discovery Through Human Alignment arXiv:2608.21057v1 Announce Type: new Abstract: Agentic large language model (LLM) systems are reshaping scientific workflows in chemistry and drug discovery, but evaluating their open-ended, tool-augmented outputs remains a fundamental bottleneck. Reference-based metrics such… 12 arXiv — Machine Learning research 6d ago aiXamine: Unified Black-Box Evaluation of Cross-Dimensional Trade-offs in LLM Safety, Security, and Privacy arXiv:2608.20554v1 Announce Type: cross Abstract: The critical failure modes in deployed large language models (LLMs) are cross-dimensional: a model can score 99.3 in safety alignment while refusing one in three benign queries, or improve across every capability metric while… 29 arXiv — NLP / Computation & Language research 6d ago When Vocabulary Comprehension Fails Clinical Reasoning: Evaluating Therapy Bots' Safety Risks for Generation Alpha arXiv:2608.20345v1 Announce Type: new Abstract: Conversational AI systems have become informal mental health support resources for Generation Alpha (Gen Alpha, born 2010-2024), with 13.1% of U.S. adolescents (5.4 million) using generative AI for mental health advice. While these… 5 arXiv — NLP / Computation & Language research 6d ago Scaling Unsupervised Word Alignment to Documents via Structural Constraints arXiv:2608.21023v1 Announce Type: new Abstract: Word alignment has traditionally been studied between sentences, but many cross-lingual tasks increasingly require correspondences across full documents. While recent multilingual embedding models can encode long inputs, we show… 27 arXiv — NLP / Computation & Language research 6d ago MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation arXiv:2601.06519v2 Announce Type: replace Abstract: Biomedical retrieval-augmented generation (RAG) can ground LLM answers in medical literature, yet long-form outputs often contain isolated unsupported or contradictory claims with safety implications. We introduce… 15 Page 1 of 10 · 500 articles Older →
