News / #safety Tag Safety + alignment 500 articles archived under #safety · RSS Sign in to follow Ars Technica — AI news-outlet 1mo ago ChatGPT and Reddit now face EU's toughest online safety rules Explosive growth comes with a new regulatory burden in the European Union. 5 Hugging Face Daily Papers research 1mo ago Language Chain in Alignment: Cross-lingual Ranking Preference Optimization Abstract Cross-lingual Ranking Preference Optimization improves multilingual alignment by transferring English preference knowledge to target languages through hierarchical ranking optimization. Generated by thinkingmachines/Inkling-Small The alignment of Large Language Models… 35 OpenAI official-blog 1mo ago OpenAI supports California’s bill to advance youth AI safety OpenAI supports California SB 1119, advancing strong, age-appropriate AI safeguards for teens while preserving opportunities to learn, create, and explore. 28 arXiv — Machine Learning research 1mo ago More Data Cannot Break a Symmetry: Identifiability by Design arXiv:2608.27651v1 Announce Type: new Abstract: Unsupervised representational alignment recovers a stimulus-by-stimulus correspondence from geometry alone, but the automorphism group of the stimulus geometry bounds what any such alignment can identify, before data exist. The… 18 arXiv — Machine Learning research 1mo ago SafeStep: An Interactive Demonstration of Semantic Communication for Pedestrian Safety Monitoring arXiv:2608.27688v1 Announce Type: new Abstract: In this paper, we develop SafeStep, an interactive browser-based semantic communication platform for live pedestrian safety monitoring. SafeStep extracts pedestrian information from four live traffic-camera feeds, transmits it… 18 arXiv — Machine Learning research 1mo ago When Can Conditional Flow Matching Replace Pointwise Negative Log-Likelihood? arXiv:2608.28010v1 Announce Type: new Abstract: Flow matching enables likelihood-free training, yet alignment methods increasingly reuse conditional flow matching (CFM) losses as endpoint negative log-likelihoods (NLLs) and their old/new differences as log-likelihood ratios. We… 16 arXiv — NLP / Computation & Language research 1mo ago The Effect of Emotional Context on Large Language Models' Endorsement of Premature Decisions: Comparing Emotional Vulnerability Across Six Commercial Models arXiv:2608.27465v1 Announce Type: new Abstract: As large language models (LLMs) are increasingly used for everyday decision-making advice, whether a model shifts the direction of its advice according to the user's emotional state has become an important safety problem. We test… 10 arXiv — NLP / Computation & Language research 1mo ago Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluation arXiv:2608.28508v1 Announce Type: new Abstract: Forced alignment evaluation typically requires manually annotated timestamps, limiting large-scale and multilingual analysis. We introduce two corpus-level metrics based on self-supervised (SSL) speech representations for… 6 arXiv — NLP / Computation & Language research 1mo ago AI Alignment through a Game-theoretic Lens: A Survey arXiv:2608.27910v1 Announce Type: cross Abstract: As large language models and increasingly capable AI agents are deployed in high-risk settings, aligning them with complex human values has become a central challenge. Existing alignment methods, while effective in improving… 34 arXiv — NLP / Computation & Language research 1mo ago Speculative Probing: LLM Monitoring at Speculative-Decoding Cost arXiv:2608.28099v1 Announce Type: cross Abstract: Real-time classification during language model inference is valuable for safety filtering, behavioral analysis, and model monitoring, but current approaches force a trade-off between accuracy and efficiency. Hidden-state probes… 20 arXiv — NLP / Computation & Language research 1mo ago When Robots Mishear Us: Mapping the Safety Risks of Voice-Controlled Embodied AI arXiv:2608.28518v1 Announce Type: cross Abstract: We investigate whether automatic speech recognition (ASR) errors in user input can lead to unsafe outputs from Embodied AI (EAI) models. We find that ASR errors can lead to harmful instructions being accepted and executed by EAI… 38 arXiv — NLP / Computation & Language research 1mo ago Steering Multimodal Large Language Models Decoding for Context-Aware Safety arXiv:2509.19212v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in real-world applications, yet their ability to make context-aware safety decisions remains limited. Existing methods often fail to balance oversensitivity… 13 Hugging Face Daily Papers research 1mo ago StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing Abstract StepGuard is a step-level guard model that audits agent actions before execution, trained via automatic trajectory generation and balanced reinforcement learning to reduce attacks with minimal utility loss. Generated by thinkingmachines/Inkling-Small LLM-based agents… 27 Ars Technica — AI news-outlet 1mo ago Meta makes AI glasses slightly less creepy with limit on nonconsensual recording Meta fixes AI glasses to stop recording any time users cover up the safety light. 19 Hugging Face Daily Papers research 1mo ago Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher Abstract Self-OPD eliminates task-specific teachers in flow matching by using self-explored stochastic branches and normalized advantages to optimize the velocity field for multi-objective alignment. Generated by thinkingmachines/Inkling-Small On-policy distillation (OPD), which… 26 arXiv — Machine Learning research 1mo ago NeuronFuzz: Safety Neuron Guided Fuzzing for LLM Safety Evaluation arXiv:2608.26222v1 Announce Type: new Abstract: Safety evaluation is critical for assessing whether aligned Large Language Models (LLMs) remain robust against jailbreak attacks. Existing automated testing methods, however, largely rely on response-level feedback: each candidate… 12 arXiv — Machine Learning research 1mo ago Privacy Without Regret: Differentially Private Inference-Time Alignment arXiv:2608.26324v1 Announce Type: new Abstract: Best-of-N (BoN) sampling is the simplest and most widely deployed inference-time alignment strategy, but it suffers from two distinct problems: reward hacking, in which the selected response exploits errors in the proxy reward… 14 arXiv — Machine Learning research 1mo ago A Single Suffix to Break Them All: Basin-Aware Jailbreaks for Merged Model Families arXiv:2608.26506v1 Announce Type: new Abstract: Model merging enables combining multiple fine-tuned models without additional training, but its safety implications remain poorly understood. Prior work primarily attributes merging risks to unsafe constituent models, implicitly… 24 arXiv — Machine Learning research 1mo ago GRAS: Guided Reduced-Variance Proposals and Adaptive Selection for Training-Free Reward Alignment in Discrete Diffusion arXiv:2608.26585v1 Announce Type: new Abstract: Discrete diffusion models have become a strong, widely adopted class of generators for sequence data, and steering them toward a downstream reward at inference time, without any retraining, is increasingly important. Such… 24 arXiv — Machine Learning research 1mo ago Safety by Design: Realized-Cost Constraints for Contextual Bandits with Continuous Actions arXiv:2608.26755v1 Announce Type: new Abstract: Contextual bandits are a standard framework for sequential decision-making under uncertainty, with applications in clinical trials, dosage selection, recommendation systems, and autonomous systems. Safety is central in many of… 18 arXiv — Machine Learning research 1mo ago Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Traffic arXiv:2608.26860v1 Announce Type: new Abstract: Connected and automated vehicle (CAV) platooning offers a promising approach to improving road safety and traffic capacity. However, platoon control in real-world traffic is challenging due to uncertainty and heterogeneous driving… 15 arXiv — NLP / Computation & Language research 1mo ago DeflectBench: A Benchmark for Evaluating Rhetorical Fallacy Generation in LLMs arXiv:2608.26119v1 Announce Type: new Abstract: Whether large language models can be prompted to generate rhetorical fallacies on demand, and whether current safety post-training constrains this behavior, has received less attention than the related question of detecting… 30 arXiv — NLP / Computation & Language research 1mo ago SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning arXiv:2608.26550v1 Announce Type: new Abstract: Reinforcement learning-based knowledge distillation has the potential to transfer complex reasoning from teacher to student models, yet it currently faces a critical dilemma: researchers must choose between sparse outcome-based… 34 arXiv — NLP / Computation & Language research 1mo ago Surgical Alignment in Knowledge Graph Training for Clinical Diagnosis with Large Language Models arXiv:2608.26587v1 Announce Type: new Abstract: Biomedical knowledge graphs (KGs) offer structured medical knowledge that can ground large language model (LLM) reasoning in clinical diagnosis application, yet how KG signal should be integrated into LLMs remains an open question.… 8 arXiv — NLP / Computation & Language research 1mo ago ITL: Interpretable Document Alignment with Structured Reference Frameworks arXiv:2608.27031v1 Announce Type: new Abstract: Measuring alignment between documents and structured reference frameworks requires identifying conceptual evidence distributed throughout the text and reporting it through measures that are quantitative, interpretable, and… 19 arXiv — NLP / Computation & Language research 1mo ago Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations? arXiv:2608.27115v1 Announce Type: new Abstract: Cross-lingual alignment in multilingual language models is typically attributed to joint training: shared parameters, mixed-language batches, or explicit alignment objectives. We ask whether monolingual models trained on… 8 arXiv — NLP / Computation & Language research 1mo ago STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translation arXiv:2608.27161v1 Announce Type: new Abstract: Large Language Models (LLMs) have enabled a shift from sentence-level to document-to-document (Doc2Doc) machine translation, promising improved global coherence. However, document-to-document generation in a single pass frequently… 11 Hugging Face Daily Papers research 1mo ago PAWBench: How Far Are We from Probabilistically Aligned World Modeling? Abstract The study formalizes probabilistic alignment for world models, introduces PAWBench and PAWEval to evaluate video generators as stochastic samplers, and finds current models fail to match reference behavior distributions. Generated by thinkingmachines/Inkling-Small… 8 Hugging Face Daily Papers research 1mo ago FIRM-Video: Check Before You Score for Reliable Text-to-Video Reward Modeling Abstract FIRM-Video uses checklist-driven verification of temporal visual evidence to build reliable reward models for text-to-video evaluation and alignment. Generated by thinkingmachines/Inkling-Small Reliable reward models are essential for text-to-video evaluation and… 32 arXiv — Machine Learning research 1mo ago Multi-Modal Anomaly Detection: A Survey arXiv:2608.24937v1 Announce Type: new Abstract: Multi-Modal Anomaly Detection (MMAD) detects rare abnormal events from heterogeneous data sources and is increasingly used in safety- and reliability-critical applications such as industrial inspection and cybersecurity. Yet the… 9 arXiv — NLP / Computation & Language research 1mo ago Learning Mixtures of Plackett-Luce Models for Multi-Objective Alignment arXiv:2608.25200v1 Announce Type: cross Abstract: We consider the problem of learning a mixture of $k$ Plackett-Luce models given multi-way ranking responses from annotators that may represent heterogeneous underlying preferences. This problem has many applications in AI… 30 arXiv — Machine Learning research 1mo ago Refusal geometry reflects refusal training: diverse refusal prefixes can raise stable rank and weaken refusal vector ablation attacks arXiv:2608.25390v1 Announce Type: new Abstract: Refusal training protects AI models from jailbreaks by training models to decline unsafe queries, reducing the risk of misuse. Recent work finds that refusal behavior in aligned language models can be mediated by a single… 32 arXiv — NLP / Computation & Language research 1mo ago Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal arXiv:2608.24988v1 Announce Type: new Abstract: Activation steering can be embedded directly into a language model's weights, shaping behaviour without inference-time intervention and offering a way to encode alignment prior to release. However, models are routinely fine-tuned… 28 arXiv — NLP / Computation & Language research 1mo ago Controllable Affective Generation via Latent Vector Steering arXiv:2608.25569v1 Announce Type: new Abstract: Large Language Models (LLMs) often produce emotionally flattened responses after alignment, limiting their effectiveness in affect-sensitive applications. In this paper, we propose EmoVec, a lightweight framework for controllable… 20 arXiv — NLP / Computation & Language research 1mo ago Localize-Then-Decide Guarantees for LLM Judgments arXiv:2608.25824v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as evaluators to assess output quality and preference alignment, yet providing reliable guarantees of agreement with human judgments remains challenging. Recent work introduces… 21 arXiv — NLP / Computation & Language research 1mo ago RefLAM: A Reference-Grounded Line Annotation Pipeline for Historical Arabic Manuscripts arXiv:2608.25140v1 Announce Type: cross Abstract: Existing approaches to building line-level Arabic handwritten-text-recognition (HTR) training data either rely on fully manual annotation, which does not scale, or on automatic OCR-to-reference alignment methods not yet extended… 33 arXiv — NLP / Computation & Language research 1mo ago Formal, Executable and Explainable Runtime Monitoring of Spoken Air Traffic Control Operational Procedures arXiv:2608.25926v1 Announce Type: cross Abstract: Air traffic control procedures are executed through spoken exchanges between controllers and pilots. These interactions are essential to the safety of air transportation: failures in their execution can create severe operational… 21 Hugging Face Daily Papers research 1mo ago When "Must" Becomes "Maybe": Constraint Weakening in LLM Agent Workflows Abstract Multi-stage LLM workflows lose operational constraints when intermediate artifacts transform binding prerequisites into non-binding context, causing safety failures despite preserved content. Generated by thinkingmachines/Inkling-Small Large language model (LLM) agents… 32 Hugging Face Daily Papers research 1mo ago Automata from Agent Traces: Failure and Next-Step Prediction Abstract LLM agent traces are compressed into compact finite-state machines that enable accurate next-step and failure prediction for safety auditing and runtime monitoring. Generated by thinkingmachines/Inkling-Small LLM-based agents execute multi-step tasks, but their… 23 arXiv — Machine Learning research 1mo ago Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Edge-Deployable Virtual Sensing arXiv:2608.23987v1 Announce Type: new Abstract: Virtual sensing enables digital twins and safety-critical systems to reconstruct and forecast spatial-temporal physics in real time. However, conventional computational and data-driven methods often face challenges in… 14 arXiv — Machine Learning research 1mo ago Joint Distribution Alignment for Universal Domain Adaptation arXiv:2608.24429v1 Announce Type: new Abstract: Unsupervised domain adaptation (UDA) has been widely concerned in the fields of machine learning, pattern recognition, and computer vision. Traditional UDA learning usually assumes that the label spaces of the source and target… 17 arXiv — Machine Learning research 1mo ago $(\text{DNN})^2$: Doubly Non-Negative Relaxations for Deep Neural Networks arXiv:2608.24743v1 Announce Type: new Abstract: Existing linear program (LP) and semidefinite program (SDP) relaxations for rectified linear unit (ReLU) neural network (NN) verification yield overly-conservative safety guarantees due to significant relaxation gaps. While the… 6 arXiv — Machine Learning research 1mo ago Automata from Agent Traces: Failure and Next-Step Prediction arXiv:2608.23670v1 Announce Type: cross Abstract: LLM-based agents execute multi-step tasks, but their behavioral structure remains opaque: long unstructured traces resist the safety auditing and runtime monitoring that deployment requires. Existing approaches operate per-trace… 5 Hugging Face Daily Papers research 1mo ago On-Policy Self-Distillation in Diffusion Models Abstract DiffusionOPSD uses on-policy self-distillation to turn image-level rewards into explicit intermediate targets for diffusion models, improving alignment efficiency and enabling separate analysis of target construction and policy fitting. Generated by… 16 OpenAI official-blog 1mo ago The Hugging Face incident and the road ahead OpenAI shares findings from the Hugging Face security incident and the steps we’re taking to strengthen AI model security, monitoring, and alignment. 21 Hugging Face Daily Papers research 1mo ago GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture Abstract GigaBrain-0.7 is a vision-language-action model that improves embodied generalization via a three-system architecture, large-scale heterogeneous pretraining, and joint alignment training. Generated by thinkingmachines/Inkling-Small Vision-language-action (VLA) models… 31 arXiv — NLP / Computation & Language research 1mo ago Evaluation Awareness in Language Models: Representation, Verbalization, and Control arXiv:2608.21766v1 Announce Type: new Abstract: Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their behavior in deployment. This assumption can fail, should models infer that they are… 16 arXiv — NLP / Computation & Language research 1mo ago No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios arXiv:2608.21775v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in real-world applications, yet they remain vulnerable to generating harmful content. From adversarial jailbreaks that bypass safety filters to implicit hate that evades… 30 arXiv — NLP / Computation & Language research 1mo ago BanglaVeilGuard: Cross-Script Safety Benchmarking and Lightweight Guardrails for Bangla Large Language Models arXiv:2608.21880v1 Announce Type: new Abstract: Bangla large language model (LLM) safety is difficult to evaluate with English-centric or standard-script benchmarks because Bangla users routinely write across scripts, spellings, code-mixed forms, and regional registers. This… 4 arXiv — NLP / Computation & Language research 1mo ago Text-Anchored Semantic Perturbations for Transferable Jailbreak Attacks on Multimodal Large Language Models arXiv:2608.22312v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language interaction, yet their safety alignment remains vulnerable to jailbreak attacks. A key challenge is that safety behavior learned in the… 13 Page 9 of 10 · 500 articles ← Newer Older →
