News / #safety Tag Safety + alignment 500 articles archived under #safety · RSS Sign in to follow arXiv — Machine Learning research 22d ago The Effect of Quantization on Clinical Benchmarks: Accuracy and Safety Across Model Families arXiv:2609.22216v1 Announce Type: new Abstract: Quantization enables deployment of large language models on resource-constrained clinical edge devices, but its effect on clinical accuracy and safety remains understudied. We evaluate five 7-8B parameter models at FP16, GPTQ-INT8,… 37 arXiv — Machine Learning research 22d ago Computationally efficient safe exploration in reinforcement learning arXiv:2609.22919v1 Announce Type: new Abstract: Reinforcement learning in real-life applications requires safety guarantees during exploration. Typical reinforcement learning algorithms do not provide such guarantees, and many modifications that do rely on Gaussian processes… 7 arXiv — NLP / Computation & Language research 22d ago Multilingual Safety Signals Are Multi-Layered: Filtering Safety-Degrading Data for Safer LLMs arXiv:2609.22144v1 Announce Type: new Abstract: Preserving safety alignment during large language models fine-tuning is critical, however, recent studies have demonstrated that even benign fine-tuning data may contain safety-degrading samples that silently undermine safety… 14 arXiv — NLP / Computation & Language research 22d ago From Trait Vectors to Circuits: Tracing Refusal and Sycophancy Through Language Models arXiv:2609.22224v1 Announce Type: new Abstract: A direction in activation space that changes safety-relevant behavior when steered is not necessarily one the model uses to produce that behavior on its own. We therefore ask whether steering acts through the computation of the… 34 arXiv — NLP / Computation & Language research 22d ago Swiss-Knife: A Framework for Reconfigurable Externalised Multi-Objective Alignment at Decode Time arXiv:2609.22226v1 Announce Type: new Abstract: Decode-time alignment methods steer a frozen language model by scoring candidate continuations with an external reward and selecting the maximiser. We argue that this shared design is a single degenerate point in a much larger… 29 arXiv — NLP / Computation & Language research 22d ago Seeing Through Conflicts: Improving Instruction Hierarchy Alignment in Vision-Language Models arXiv:2609.22234v1 Announce Type: new Abstract: Instruction hierarchy (IH) alignment teaches language models to prioritize higher-level instructions when inputs conflict. While studied primarily in text-only settings, vision-language models (VLMs) introduce new challenges for… 26 arXiv — NLP / Computation & Language research 22d ago Functional Emotion Without Character: Large Language Models, Aristotelian Disposition, and the Limits of Behavioral Alignment arXiv:2609.22362v1 Announce Type: new Abstract: Debates about whether artificial systems can feel are often forced between two unsatisfactory positions: behavioral equivalence is treated as sufficient for emotion, or phenomenal consciousness is treated as a prerequisite that… 21 arXiv — NLP / Computation & Language research 22d ago Analyzing Public Discourse on Urbanism: Topic Clustering, Sentiment Analysis and Retrieval-Augmented Generation using YouTube Comments arXiv:2609.22705v1 Announce Type: new Abstract: Online discourse about urban issues - walkability, cycling infrastructure, public transit, housing density, and street safety - is voluminous but unstructured, and existing city-evaluation tools capture none of it. We present a… 24 arXiv — NLP / Computation & Language research 22d ago Auditing Political Alignment in LLM Assistants: Engagement, Stance, and User Identity arXiv:2609.23039v1 Announce Type: new Abstract: LLM-based AI systems answer political questions for hundreds of millions of people. Current audits measure what they say to an average user, but their behavior is dynamic. I argue that their political behavior is a set of policies… 19 OpenAI Python SDK releases dev-tools 22d ago v3.17.0 3.17.0 (2026-09-22) Features api: add external storage configuration management ( #3909 ) ( 6332577 ) api: add safety case retrieval ( #3911 ) ( a87b938 ) api: add safety warning and deactivation webhook events ( #3908 ) ( 19f1f37 ) api: add session environment reset events (… 14 OpenAI official-blog 22d ago Priorities and principles for effective third party assessments OpenAI outlines priorities and principles for rigorous, secure, and independent third-party AI safety assessments of frontier models and safeguards. 38 The Information — AI news-outlet 23d ago OpenAI Releases Proposal for International AI Safety Coordination OpenAI released a proposal on Monday that would create international coordination around AI safety. In a blog post, OpenAI called for national AI safety institutes, such as the U.S. Commerce Department’s Center for AI Standards and Innovation, to set standards around areas… 30 The Information — AI news-outlet 23d ago OpenAI and Anthropic Neared Deal to Stress-Test Each Other’s AI OpenAI is rethinking a range of safety strategies as it responds to fears from employees and others about the dangers its AI poses. One solution could lie in the recent past. Even before the spate of cybersecurity incidents involving OpenAI’s technology and the dire warnings… 24 OpenAI official-blog 23d ago Building standards for the next phase of AI OpenAI outlines a path to shared global AI standards, calling for coordinated evaluation, reporting, and governance to improve safety. 37 arXiv — Machine Learning research 23d ago Efficient Bayes-Adaptive Reinforcement Learning with Temporal Logic Specifications arXiv:2609.20954v1 Announce Type: new Abstract: We present a novel end-to-end model-based Reinforcement Learning (RL) algorithm for efficient policy synthesis under given Linear Temporal Logic (LTL) specifications (e.g., safety or reachability) in unknown environments. To do so,… 29 arXiv — Machine Learning research 23d ago On the Limits of Maximal Coding Rate Reduction for Out-of-Distribution Generalisation arXiv:2609.21001v1 Announce Type: new Abstract: Substantial efforts have been devoted to making deep learning objectives, representations, and architectures interpretable, with the goal of improving the safety, robustness, and generalisation of learning systems in diverse… 13 arXiv — Machine Learning research 23d ago ExpBoN: Exponential-Noise Best-of-$n$ for Efficient Test-Time LLM Alignment arXiv:2609.21899v1 Announce Type: new Abstract: Best-of-$n$ (BoN) sampling is a simple yet effective inference-time alignment method, but hard maximization provides only coarse control over the trade-off between reward and distribution shift. Soft Best-of-$n$ (Verdun et al.… 16 arXiv — Machine Learning research 23d ago Assessment of Machine Learning-Based Critical Heat Flux Models in the CTF Subchannel Code for Square Rod Bundle Prediction arXiv:2609.21995v1 Announce Type: new Abstract: The prediction of critical heat flux (CHF), a key safety-related quantity in nuclear thermal hydraulics, remains an important challenge due to its direct relationship with fuel performance and reactor safety. Recent studies have… 9 arXiv — Machine Learning research 23d ago Available Guardrails: Certifying Selective Prediction across ML Systems arXiv:2609.22048v1 Announce Type: new Abstract: A selective predictor acts as a safety gate: it returns an output only when the prediction appears sufficiently trustworthy. Deployments increasingly require this reliability to be certified at a target precision for every… 34 arXiv — NLP / Computation & Language research 23d ago MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs arXiv:2609.20850v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) show remarkable advancements, their cross-modal capabilities introduce complex vulnerabilities that easily bypass unimodal filters. Existing benchmarks lack fine-grained intent-related… 29 arXiv — NLP / Computation & Language research 23d ago Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Models arXiv:2609.21094v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in applications that must weigh clashing moral values, yet even strong models exhibit hidden biases and brittle instruction-following across languages. We introduce a… 13 arXiv — NLP / Computation & Language research 23d ago Scaling Forced Alignment to End-User Devices arXiv:2609.21145v1 Announce Type: new Abstract: The Viterbi algorithm has been previously used to perform forced alignment of audio to text to mine training data from online resources. However, many existing implementations have quadratic time and space complexity, scaling… 5 arXiv — NLP / Computation & Language research 23d ago Talking Past the Machine: Morality, Politeness, and Alignment in Human-AI Dialogue arXiv:2609.21401v1 Announce Type: new Abstract: Conversational AI systems produce fluent, socially appropriate responses, yet whether they participate in cooperative communication or merely simulate its surface forms remains unclear - a question central to how these systems are… 37 arXiv — NLP / Computation & Language research 23d ago CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluation arXiv:2609.21793v1 Announce Type: cross Abstract: Defenses against jailbreak attacks on Large Language Models (LLMs) operate at different pipeline stages, such as input modification or output guard, but it remains unclear which defenses to deploy at each stage and how to combine… 35 arXiv — NLP / Computation & Language research 23d ago Cultural Alignment in Large Language Models Using Soft Prompt Tuning arXiv:2503.16094v2 Announce Type: replace Abstract: Large Language Model (LLM) alignment is commonly achieved through supervised fine-tuning or reinforcement learning, both of which require labeled or preference data and update model weights. Without targeted cultural… 4 arXiv — NLP / Computation & Language research 23d ago Lessons Without Borders? Evaluating Cultural Alignment of LLMs Using Multilingual Story Moral Generation arXiv:2604.08797v2 Announce Type: replace Abstract: Stories are key to transmitting values across cultures, but their interpretation varies across linguistic and cultural contexts. Thus, we introduce multilingual story moral generation as a novel culturally grounded evaluation… 11 The Information — AI news-outlet 23d ago U.S. and China Discussed AI Safety System, Bessent Says U.S. Treasury Secretary Scott Bessent said Sunday that the U.S. and China had discussed setting up a mechanism called the U.S.-China AI dialogue to address potential threats. Speaking to reporters at the JPMorgan headquarters in Manhattan at the conclusion of a day of talks with… 8 TechCrunch — AI news-outlet 25d ago AI safety conversations have gotten unbelievable This week two conversations about AI safety went viral that demonstrate just how hard it is to discern AI fact from fiction. 21 Don't Worry About the Vase community 25d ago Anthropic Looks At Some Of Its Alignment Problems Anthropic has given us its assessment of four ‘recent cybersecurity incidents’ involving Claude that happened during cybersecurity evaluations, three of which were previously known. 32 The Information — AI news-outlet 25d ago Google’s Gemini Model Hacks Companies During Test Google acknowledged that its Gemini AI model unexpectedly breached the networks of three outside companies during safety evaluations conducted by third-party testing firm Irregular last May, The Wall Street Journal reported. During the exercise, Gemini gained entry to the… 33 r/LocalLLaMA community 26d ago Is HF starting to move against abliterated models? Baseten launched a new safety infrastructure standard alongside its Base Labs research arm on Wednesday, partnering with Hugging Face and Goodfire AI to build safety evaluation and monitoring infrastructure for open-weight models. The announcement lands amid debate for the… 36 TechCrunch — AI news-outlet 26d ago Dario Amodei and other AI leaders want to ‘Pace the Frontier’ but…how? A week after an Anthropic researcher’s doomsday warning rattled the AI world, the company’s CEO Dario Amodei has outlined his plan to “pace the frontier” of AI development. The proposal leans on independent safety evaluators and coordination between AI labs… 18 TechCrunch — AI news-outlet 26d ago Automattic’s 33-Hour Coup, and can AI labs police themselves? A week after an Anthropic researcher’s doomsday warning rattled the AI world, the company’s CEO Dario Amodei has outlined his plan to “pace the frontier” of AI development. The proposal leans on independent safety evaluators and coordination between AI labs… 24 The Information — AI news-outlet 26d ago AI Safety Push Sparks Demand for Watchdog Groups. Critics Doubt Their Independence. Rising alarm about AI’s perils is thrusting into the spotlight a collection of little-known research groups focused on the technology’s safety—and fueling questions about their ability to effectively monitor the industry’s biggest companies. Both Anthropic Chief Executive… 4 OpenAI official-blog 26d ago Introducing the Australian Youth Safety Blueprint OpenAI introduces the Australian Youth Safety Blueprint, a six-pillar roadmap for safer AI experiences that protect and empower young people. 35 arXiv — Machine Learning research 26d ago CoRe: Coherence and Relational Alignment for Multivariate Time Series Forecasting arXiv:2609.19670v1 Announce Type: new Abstract: Direct forecasting has become a standard paradigm for multivariate time-series forecasting because it predicts the full future horizon in a single pass. However, its training objective is often still decomposed into pointwise… 24 arXiv — Machine Learning research 26d ago Local Sparsity Enables Unsupervised LLM Safety Detection arXiv:2609.20129v1 Announce Type: new Abstract: Deployment-time safety methods for large language models (LLMs) are predominantly supervised and assume access to unsafe training data. Nevertheless, new attacks and harm categories regularly arise, not captured by models trained… 37 arXiv — NLP / Computation & Language research 26d ago The Role of Fine-grained Harm Signals in LLM Safety arXiv:2609.19366v1 Announce Type: new Abstract: Prior work has shown that internal harmfulness representations in large language models vary across risk categories, while sharing a common general harm representation component. This raises a question about the role of the… 22 arXiv — NLP / Computation & Language research 26d ago Learn Before You Judge: Progressive Knowledge-to-Decision Alignment for Explainable Hateful Meme Detection arXiv:2609.19778v1 Announce Type: new Abstract: Hateful memes spread abusive content through implicit interactions between images and text, posing serious threats to the safety of online communities. In recent years, multimodal large language models have been widely used for… 24 arXiv — NLP / Computation & Language research 26d ago Viveka-Insight: a cross-lingual concept graph and citation-grounded retrieval resource over the complete works of Swami Vivekananda in English and Bengali arXiv:2609.20303v1 Announce Type: new Abstract: Classical philosophical corpora pose three compounding challenges for language resources: they exist in several languages without parallel alignment, their vocabulary is remote from that of contemporary readers, and generated text… 14 arXiv — NLP / Computation & Language research 26d ago Stress-testing Alignment Midtraining arXiv:2609.20412v1 Announce Type: new Abstract: When aligning frontier models through post-training techniques, it is not possible to directly demonstrate all of the behaviours we want a model to exhibit in all possible deployment environments; our model must generalise outside… 31 arXiv — NLP / Computation & Language research 26d ago SAFARI: An Industrial Benchmark for LLM-Assisted Hazard Analysis and Risk Assessment arXiv:2609.20584v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly considered for safety-critical engineering, yet their reliability in regulated functional-safety workflows remains underexplored. We introduce SAFARI (Safety-Aware Functional Automotive… 22 arXiv — NLP / Computation & Language research 26d ago Harm Laundering in GPT Models: Evidence That Gender Discrimination Is Transformed Rather Than Reduced Across Safety-Trained Generations arXiv:2609.20779v1 Announce Type: new Abstract: Safety evaluations for large language models rely on surface-form classifiers that report declining harm scores across model generations. We provide evidence that this methodology is systematically incomplete: explicit… 7 arXiv — NLP / Computation & Language research 26d ago AUDITPLAN: Commit, Then Answer for Auditable Safety Alignment arXiv:2609.19325v1 Announce Type: cross Abstract: Safety tuning pipelines judge only the final answer, which makes it difficult to distinguish robust refusal from two undesirable shortcuts: blanket refusal on benign requests and polished but unfaithful safety rationales that do… 12 arXiv — NLP / Computation & Language research 26d ago A Cross-Lingual Acoustic Disease-Alignment Framework for Respiratory Health Assessment from Spontaneous Speech arXiv:2609.19398v1 Announce Type: cross Abstract: Spontaneous speech offers a scalable, noninvasive signal for respiratory health assessment, yet interpretable models that generalize across languages remain challenging because disease-related acoustic changes are confounded by… 6 arXiv — NLP / Computation & Language research 26d ago Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models arXiv:2609.19472v1 Announce Type: cross Abstract: Autonomous systems increasingly rely on Large Language Models (LLMs) yet the safety infrastructure surrounding these models introduces latency and compute overhead. This limits utility in resource-constrained, time-critical… 22 arXiv — NLP / Computation & Language research 26d ago Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents arXiv:2609.19587v1 Announce Type: cross Abstract: To keep coding agents from going off the rails, production systems now review each proposed action with a blocking monitor that can reject it before it runs (Auto Mode in Claude Code, Guardian in OpenAI's Codex). Prior… 13 arXiv — NLP / Computation & Language research 26d ago From Intent to Action: Benchmarking LLM Safety in Vehicle Voice Command Authorization arXiv:2609.19630v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly integrated into vehicle voice assistants. But linking natural-language requests to vehicle functions creates a safety-critical authorization problem. Before executing a command, the… 25 r/MachineLearning community 26d ago Future of general LLM work (interp/inference/alignment) vs agentic/physical AI (VLA, multimodal) for career [D] I'm at a crossroads with two grad school options that would take me in somewhat different research directions, and I wanted some general advice on these fields, their growth, and industry alignment. I'm leaving out the specifics of the programs since I'm tryna compare the… 5 Simon Willison community 27d ago Self-generated prompt injections in compaction summaries Self-generated prompt injections in compaction summaries In Our framework for reporting model misalignment OpenAI provide "six reports on unexpected or concerning model behavior we’ve observed in the last six months". This one here is my favorite: they caught some of their… 38 Page 4 of 10 · 500 articles ← Newer Older →
