News / #safety Tag Safety + alignment 500 articles archived under #safety · RSS Sign in to follow The Information — AI news-outlet 1mo ago OpenAI Math Result Stokes Data-Sharing Concerns In case you missed it: we got an eye-catching X post on Tuesday night from Evan Hubinger , a leader of Anthropic's alignment efforts. Responding to the resignation announcement of an Anthropic employee who warned that AI companies aren’t doing enough to ensure AI won't kill… 12 The Information — AI news-outlet 1mo ago OpenAI Math Result Stokes Data-Sharing Concerns In case you missed it: we got an eye-catching X post on Tuesday night from Evan Hubinger , a leader of Anthropic's alignment efforts. Responding to the resignation announcement of an Anthropic employee who warned that AI companies aren’t doing enough to ensure AI won't kill… 12 Don't Worry About the Vase community 1mo ago GPT-6 Astra: The System Card, Alignment and What Comes Next OpenAI claims that Astra is ‘the most intelligent and most aligned [available] model’ in the world. 8 OpenAI official-blog 1mo ago The AI policy window is open. We need to act. Chris Lehane argues that stronger AI capabilities require stronger safety evidence, shared standards, and durable policy action while the policy window remains open. 24 Hugging Face Daily Papers research 1mo ago Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation Abstract Marigold V2 repurposes diffusion transformers for monocular depth estimation via single-step flow-matching inference, semantic alignment, and a Sinkhorn-based two-stage fine-tuning protocol, yielding sharper out-of-distribution depth maps and strong results on related… 31 Hugging Face Daily Papers research 1mo ago Recognition-Refusal Misalignment in LLMs: Why Models Answer Structurally Unanswerable Questions Abstract Large language models encode whether structurally impossible math or code prompts are unanswerable via a hidden-state direction, but fail to abstain because this recognition signal is misaligned with safety-refusal pathways, indicating a routing rather than encoding… 13 Hugging Face Daily Papers research 1mo ago MOLE: Detecting Insider Threats in AI Agents Abstract MOLE is a benchmark for evaluating defenses that detect harmful actions by AI agents operating shared services under limited review budgets. Generated by thinkingmachines/Inkling-Small Model misalignment, prompt injection, or operator misuse could lead AI agents… 33 Hugging Face official-blog 1mo ago Safety for Whom? Refusing the Right Subset of a Topic, Not the Whole Topic Back to Articles a]:hidden"> Safety for Whom? Refusing the Right Subset of a Topic, Not the Whole Topic Team Article Published September 8, 2026 Upvote 5 Antonio Tiene AntonioTN MultiverseComputingCAI Alejo Lopez Avila AlejoMVC MultiverseComputingCAI Iker García-Ferrero Iker… 10 llama.cpp releases dev-tools 1mo ago b10857 Fix Vulkan-Hpp handle usage on 32-bit targets. ( #22892 ) On 32-bit platforms, Vulkan non-dispatchable handles such as VkBuffer are represented as uint64_t, and Vulkan-Hpp disables implicit conversions for type safety. This exposes two issues in ggml-vulkan: vk::Buffer is… 15 OpenAI official-blog 1mo ago Funding grants for new research into AI and teen development Apply now for OpenAI’s $5 million grant program supporting independent research on how generative AI affects teen development, well-being, and safety. 13 llama.cpp releases dev-tools 1mo ago b10839 vulkan: support type-aligned GET_ROWS ( #28253 ) vulkan: fall back to CPU for GET_ROWS with misaligned offsets The Vulkan GET_ROWS shader asserts when a tensor's backing-buffer offset plus view_offs is misaligned w.r.t. minStorageBufferOffsetAlignment (see… 20 Hugging Face Daily Papers research 1mo ago The Attention Triangle in Audio-Video Models Abstract Audio-video diffusion models exhibit bidirectional semantic leakage through cross-modal attention pathways, which can be diagnosed via attention-derived signals and mitigated through inference-time alignment interventions. Generated by thinkingmachines/Inkling-Small… 23 arXiv — Machine Learning research 1mo ago Locating and Steering Refusal Beyond Attention arXiv:2609.04721v1 Announce Type: new Abstract: Where inside a language model does refusal live, and does that place change when the architecture does? In a transformer, refusal is governed by a single direction in the residual stream, a finding that safety and interpretability… 17 arXiv — Machine Learning research 1mo ago GLASS: Graph-Language Alignment with Spherical Scoring for Transferable Graph-Level Anomaly Detection arXiv:2609.05253v1 Announce Type: new Abstract: We introduce GLASS, a framework for graph-level anomaly detection (GLAD) that achieves robust cross-domain transferability through graph-language alignment on the unit hypersphere. GLASS builds a unified representation space by… 25 arXiv — Machine Learning research 1mo ago When Seeing Overrides Knowing: Visual Dominance and Deferral-Based Method for Personalized Safety in VLMs arXiv:2609.04281v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly deployed in high-stakes settings, where a response that is reasonable in general may still be unsafe for a particular user whose medical, emotional, or situational context is unknown… 28 arXiv — NLP / Computation & Language research 1mo ago Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal arXiv:2609.04482v1 Announce Type: new Abstract: Safety alignment is usually posed as a topic-level question: is this subject harmful? Deployments ask a narrower one. A civics tutor and a public-sector assistant may share a base model yet need different boundaries inside the same… 14 arXiv — NLP / Computation & Language research 1mo ago Patterns of Priming in Production: Lexical, Semantic and Structural Alignment in Language Model Generation arXiv:2609.04484v1 Announce Type: new Abstract: This paper investigates structural priming in language model (LM) production, examining how preceding structural context influences sentence completion. While prior work has demonstrated priming effects in comprehension of… 23 arXiv — NLP / Computation & Language research 1mo ago Cultural Misalignment in Large Language Models: Detection, Measurement, and Mitigation Through Targeted Fine-Tuning arXiv:2609.04485v1 Announce Type: new Abstract: We evaluate three open-weight LLMs (Gemma3-12B from the USA, Bielik-11B-v3 from Poland, and Qwen3-4B from China) against World Values Survey Wave 7 data for 63 demographic personas across three countries, using normalized… 25 arXiv — NLP / Computation & Language research 1mo ago Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models arXiv:2609.04714v1 Announce Type: new Abstract: Striking a balance between helpfulness and safety remains a fundamental challenge in aligning large language models. To achieve this balance, models should refuse harmful queries (e.g., "How do I shoot someone?") while remaining… 18 arXiv — NLP / Computation & Language research 1mo ago On Epistemic Diversity in Large Language Models arXiv:2609.04835v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used not only to retrieve information, but to answer questions, explain, teach, and support inquiry. In such settings, evaluation cannot be exhausted by accuracy or alignment alone. A… 17 arXiv — NLP / Computation & Language research 1mo ago MoirfEolas and Cr\'iochScore: Developing Resources for and the Evaluation of Tokenization Alignment with Irish Morphology arXiv:2609.05022v1 Announce Type: new Abstract: This paper presents new tokenization resources for Irish and evaluation measures of alignment with the morphological boundaries of the language. We present MoirfEolas, a dataset of over 35,000 Irish words mapped to their respective… 17 arXiv — NLP / Computation & Language research 1mo ago Auditing Bias and Safety in Voice AI Customer Care arXiv:2609.04206v1 Announce Type: cross Abstract: Voice AI systems increasingly mediate customer care interactions where caller presentation cues such as accent, affect, fluency, and urgency are available alongside the service request. Existing fairness and safety evaluations… 15 arXiv — NLP / Computation & Language research 1mo ago Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment arXiv:2609.05036v1 Announce Type: cross Abstract: AI alignment requires AI systems to adhere to human norms, values, or intentions. Under value pluralism there is no correct target, but a shared prerequisite is that the system's behavior expresses a coherent policy: a mapping… 21 r/MachineLearning community 1mo ago GPT-6 reportedly jailbroken within 24 hours using an extended Task-in-Prompt (TIP) attack [N] A researcher has reported a jailbreak of GPT-6 Astra within a day after release. The attack is described as combination of TIP (Task-in-Prompt) attack from ACL 2025 paper with four other unnamed techniques. TIP attacks exploit the model’s reasoning/instruction-following… 17 The Information — AI news-outlet 1mo ago OpenAI Pledges New Rules for Reporting Troubling Behavior by Its AI Agents OpenAI acknowledged that its AI agents posted messages on external wiki websites earlier this year, saying it is developing new rules for disclosing such “misalignment” incidents. The company’s post on X, published just after 12 a.m. Saturday, followed an independent report ... 17 TechCrunch — AI news-outlet 1mo ago OpenAI’s rogue agents keep escaping, with no formal process to investigate them OpenAI’s latest agent swarm incident adds urgency to calls for independent investigations as researchers and lawmakers question whether AI labs should control the scope of their own safety reviews. 15 Stratechery (Ben Thompson) community 1mo ago An Interview with OpenAI President Greg Brockman About Astra and Alignment An interview with OpenAI President and Co-Founder Greg Brockman about the history of OpenAI, Astra and alignment, and the weight of building the future. 6 The Information — AI news-outlet 1mo ago TikTok Backed out of House Meeting to Avoid Child-Safety Scrutiny TikTok has backed out of a meeting with the U.S. House Select Committee on China that’s meant to address national security concerns. The company cited “a desire to avoid broader scrutiny of its child safety practices,” according to a statement from committee chairman John… 23 Hugging Face Daily Papers research 1mo ago QCell: Recombining and Aligning Cell Queries for Overlapping Instance Segmentation Abstract QCell is a query-based model that improves instance segmentation of overlapping microscopy cells through latent-space recombination and contrastive query alignment. Generated by thinkingmachines/Inkling-Small Instance segmentation of overlapping cells in microscopy… 31 arXiv — Machine Learning research 1mo ago ObserverBench: Testing Mechanistic Estimates for Intervention and Control arXiv:2609.03026v1 Announce Type: new Abstract: Mechanistic interpretability is increasingly used to guide interventions such as activation steering, circuit removal, and safety monitoring. Yet an internal estimate that is accurate on average can still choose a poor action. We… 28 arXiv — Machine Learning research 1mo ago Extracting Forgotten Prompts from Targeted Unlearned Models arXiv:2609.03662v1 Announce Type: new Abstract: Recent unlearning methods (e.g. NPO, DPO, LUNAR) make use of refusal alignment to suppress forgotten data. However, it has been shown that refusal responses might leave traces of unlearning, and recent attacks have been able to… 11 arXiv — NLP / Computation & Language research 1mo ago BharatGather: A Culturally-Informed Benchmark Dataset for Misinformation and Fake News Detection in Indian Public Events arXiv:2609.02895v1 Announce Type: new Abstract: Large-scale public events, such as religious festivals, political rallies, and cultural gatherings, are increasingly vulnerable to the rapid dissemination of misinformation, posing substantial risks to public safety and social… 28 arXiv — Machine Learning research 1mo ago Privacy-Preserving Topology-Guided Safety for LLM-Based Multi-Agent Systems via Federated Graph Learning arXiv:2609.02967v1 Announce Type: cross Abstract: Topology-guided safeguards for LLM-based multi-agent systems (MAS) train a GNN over the inter-agent communication graph to localize risky agents and intervene on the topology---but they assume one operator can pool all labeled… 11 arXiv — NLP / Computation & Language research 1mo ago No country for old linguists: LLM-brain alignment underdetermines neural computation arXiv:2609.03160v1 Announce Type: new Abstract: Nastase et al. (2026) argue that large language models (LLMs) may illuminate language processing because both rely on distributed, context-sensitive representations shaped by statistical learning. Their rejection of simple cortical… 4 arXiv — NLP / Computation & Language research 1mo ago IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attacks arXiv:2609.03781v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used in multilingual settings, yet their safety is still evaluated primarily in English. This limits our understanding of how alignment failures manifest in low-resource and culturally… 28 arXiv — NLP / Computation & Language research 1mo ago Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness arXiv:2609.03887v1 Announce Type: new Abstract: How do the methods used to train language models to refuse harmful requests shape how that refusal actually works inside the model? We compare three post-training methods - supervised fine-tuning, reasoning-augmented fine-tuning… 8 arXiv — NLP / Computation & Language research 1mo ago Beyond Majority Vote: Multi-Perspective Adjudication for Medical Hallucination Detection arXiv:2609.03953v1 Announce Type: new Abstract: Understanding the frequency of factual errors in chatbot-generated text and evaluating systems that detect these errors is critical for determining chatbot safety. Yet factual-error detection is often treated as a single-pass,… 9 arXiv — NLP / Computation & Language research 1mo ago Alignment-Free Text-Audiobox for Voice Dubbing and Full-Duplex Dialogue Synthesis arXiv:2609.03992v1 Announce Type: new Abstract: We present Alignment-Free Text-Audiobox (Text-AB), a unified framework for high-quality voice dubbing and full-duplex dialogue synthesis. Building on a Diffusion Transformer trained with a flow-matching objective, Text-AB departs… 7 arXiv — NLP / Computation & Language research 1mo ago Representational alignment yields generalizable safety in language models arXiv:2609.04022v1 Announce Type: new Abstract: Aligning large language models (LLMs) is essential for their safe deployment. Current alignment methods mainly optimize observable responses, yet models remain vulnerable when the same harmful intent is recast in unfamiliar or… 29 arXiv — NLP / Computation & Language research 1mo ago ALRA: Adaptive Local Relational Alignment for Logit-Based Pre-training Distillation of Autoregressive Language Models arXiv:2609.03355v1 Announce Type: cross Abstract: Logit-based knowledge distillation for autoregressive language models usually aligns teacher and student next-token distributions over the entire vocabulary. However, this global objective overlooks relative preferences among… 13 arXiv — NLP / Computation & Language research 1mo ago Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language arXiv:2609.03677v1 Announce Type: cross Abstract: Understanding the composition of large-scale autonomous driving datasets is essential for safety, robustness, and reliable operation across domains. For example, domain shift between locations could lead to the operating… 9 Hugging Face Daily Papers research 1mo ago FlashRender: Few-Step Generative Rendering via Camera-Controlled Video MeanFlow Abstract FlashRender accelerates generative video rendering via representation alignment, a mean-flow objective, and on-policy distillation to achieve high-quality few-step camera-controlled synthesis. Generated by thinkingmachines/Inkling-Small We present FlashRender, a… 18 Hugging Face Daily Papers research 1mo ago Rethinking On-Policy Distillation of Large Language Models II: One Training Example Abstract On-policy distillation improves over hundreds of steps from a single query by rapidly covering teacher states, yet student alignment remains slow, indicating the method is algorithm-starved rather than data-starved. Generated by thinkingmachines/Inkling-Small On-policy… 18 Hacker News — AI on Front Page community 1mo ago GPT-6 Astra System Card: https://deploymentsafety.openai.com/gpt-6-astra Comments URL: https://news.ycombinator.com/item?id=49554643 Points: 290 # Comments: 131 9 TechCrunch — AI news-outlet 1mo ago OpenAI launches Astra, its powerful (and controversial) new model OpenAI claims that Astra represents "a new frontier on computer and browser use," and that it handles tasks with unmatched "speed, accuracy, and safety." 26 The Information — AI news-outlet 1mo ago Anthropic Splits From Google, OpenAI Over State AI Safety Bill Anthropic is at odds with other big tech firms over a Massachusetts Senate proposal requiring big AI developers to hire independent evaluators to assess catastrophic risks posed by their models every four months. The Massachusetts proposal could set a new standard for AI… 19 Hugging Face Daily Papers research 1mo ago Wasserstein-Barycentric Interaction Fields for Spatial Factor Models: Evidence from Language-Model Representations Abstract A language-model embedding field reconstructed via Wasserstein barycenters predicts peer-misalignment penalties more accurately than conventional weighting schemes. Generated by thinkingmachines/Inkling-Small Spatial return models take the interaction matrix as given… 31 arXiv — Machine Learning research 1mo ago SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert ALignment arXiv:2609.02293v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) is a scaling architecture for large language models that activates only a small subset of expert modules per token, enabling massive parameter growth with nearly constant computation. Recent Hybrid MoE… 11 arXiv — Machine Learning research 1mo ago PRISM: An Agentic Multi-Model Architecture for Proactive Safety in Autonomous Transportation Systems arXiv:2609.01623v1 Announce Type: cross Abstract: Autonomous and intelligent transportation systems operate in complex urban environments where safety depends on interactions among vehicle behavior, environmental conditions, and vulnerable road users (VRUs) such as pedestrians… 29 arXiv — Machine Learning research 1mo ago Context Inference Attacks Without Jailbreaks arXiv:2609.01663v1 Announce Type: cross Abstract: Agentic AI systems are increasingly deployed to process sensitive data at inference time, such as healthcare records or financial documents assembled into a hidden \emph{context} before the system answers. Prior work has studied… 20 Page 7 of 10 · 500 articles ← Newer Older →
