News / #funding Tag Funding 500 articles archived under #funding · RSS Sign in to follow arXiv — Machine Learning research 29d ago JumpStart Your Policy Learning with Lessons from 160,000 Training Runs arXiv:2609.13730v1 Announce Type: new Abstract: Reliable progress in offline policy learning depends on careful reporting, well-tuned baselines, and evaluation across diverse conditions. Prior work has shown that results can be sensitive to reporting choices, hyperparameter… 10 arXiv — Machine Learning research 29d ago Diagnosing Temporal Misalignment in Multichannel Time-Series Classification with Minimum Description Length arXiv:2609.14595v1 Announce Type: new Abstract: Multichannel time-series classification commonly assumes synchronized sensor streams, although latency, clock drift, and preprocessing can introduce relative delays during data collection or after deployment. Existing… 25 arXiv — NLP / Computation & Language research 29d ago Hindsight Bias in Clinical Temporal Reasoning: How Future Data Exposure Affects Large Language Model Judgment arXiv:2609.13454v1 Announce Type: new Abstract: Clinical decisions are prospective, but clinical language models are often evaluated on retrospective records that reveal the final diagnosis, treatment response, and outcome. Such evaluations may reward the use of future… 19 arXiv — NLP / Computation & Language research 29d ago From Token Probabilities to Semantic Constraints: Towards Declarative Probabilistic Evaluation of Language Models arXiv:2609.13520v1 Announce Type: new Abstract: While Large Language Models have improved rapidly, many fundamental questions remain about how to evaluate the knowledge and reasoning abilities they acquire, and how such evaluations relate to the learning signals used in… 31 arXiv — NLP / Computation & Language research 29d ago In the Blind: Building Pseudo-References for MT Evaluation arXiv:2609.13611v1 Announce Type: new Abstract: The WMT26 General MT task evaluates systems on 10 language pairs that have no human references (neither translated from scratch nor post-edited from MT output by humans). We describe how we built the pseudo-references for these… 19 arXiv — NLP / Computation & Language research 29d ago When Consistency Does Not Mean Reliability: Evaluating Local LLM Judges Against Human Ratings arXiv:2609.13824v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to evaluate the responses of other language models. This approach, known as LLM-as-a-Judge, is faster and cheaper than human evaluation. However, a judge may produce consistent… 10 arXiv — NLP / Computation & Language research 29d ago Measuring the Cost of Variety Conflation in Multilingual MT Evaluation: Adding Mozambican Xichangana, Nyanja and Sena to FLORES+ arXiv:2609.13847v1 Announce Type: new Abstract: In this paper, we extend FLORES+ with Portuguese-source evaluation sets for three Mozambican Bantu varieties: Xichangana, Mozambican Nyanja, and Sena. We compare Xichangana with the existing Tsonga reference and Mozambican Nyanja… 10 arXiv — NLP / Computation & Language research 29d ago Mizan: A National Benchmark for Evaluating Large Language Models on Iraqi Arabic and the Iraqi Civic Context arXiv:2609.13980v1 Announce Type: new Abstract: Arabic large-language-model (LLM) evaluation has matured around Modern Standard Arabic (MSA): aggregated leaderboards such as the Open Arabic LLM Leaderboard (OALL), HELM Arabic, and BALSAM rank models across dozens of MSA tasks,… 37 arXiv — NLP / Computation & Language research 29d ago Document Topic Alignment Metrics for Evaluating Topic Models of Short-Text Public Health Communications on Social Media arXiv:2609.14256v1 Announce Type: new Abstract: Topic models are widely used to analyze public health-related social media short texts, yet their evaluation remains dominated by metrics that focus entirely on generated topics alone. There is a lack of metrics that quantitatively… 27 arXiv — NLP / Computation & Language research 29d ago E2A-Bench: Benchmarking Evidence-to-Action Reliability in Financial Chart Reasoning arXiv:2609.14302v1 Announce Type: new Abstract: Can financial vision-language models (VLMs) turn chart evidence into reliable action recommendations? Existing hallucination evaluations are mostly claim-centric; they assess whether generated statements are supported, but not… 6 arXiv — NLP / Computation & Language research 29d ago Policy Loopholes in Agent Evaluation: When Policy Ambiguity Masquerades as Agent Error arXiv:2609.14400v1 Announce Type: new Abstract: Agent benchmarks evaluate policy compliance but assume each policy determines a unique correct action. Natural-language policies can violate this assumption through silence, ambiguity, or contradiction, admitting multiple… 12 arXiv — NLP / Computation & Language research 29d ago Mind Which Bird You Favour: Parameterizing Adequacy-Fluency Balance in Meta-Evaluation of Machine Translation arXiv:2609.14795v1 Announce Type: new Abstract: There is a tradeoff in machine translation meta-evaluation between prioritizing alignment with adequacy versus fluency. The balance depends on the combination of translation systems in the meta-evaluation dataset. This system set… 31 arXiv — NLP / Computation & Language research 29d ago A primer on evaluation methods for large language models in healthcare arXiv:2609.14819v1 Announce Type: new Abstract: Large language models (LLMs) have a growing range of applications in medicine, and their evaluation is critical for ensuring they provide benefit and not harm. This evaluation can be more challenging than traditional machine… 26 arXiv — NLP / Computation & Language research 29d ago One Example Is Enough to Pass Fairness Benchmarks: Rethinking Fairness Evaluation for Aligned LLMs arXiv:2609.14860v1 Announce Type: new Abstract: Warning: This submission studies stereotypes and biases, and contains toxic and offensive examples, used for illustration purposes only. Fairness benchmarks such as BBQ have become the de facto standard for fairness evaluation… 34 Hugging Face Daily Papers research 29d ago Dream-RSI: Recursive Self-Improvement through Evolving Worlds Abstract Dream-RSI enables scalable recursive self-improvement by using historical discovery replay to evaluate exploration policies offline, reducing costly online evaluations. Generated by thinkingmachines/Inkling-Small Recursive self-improvement is becoming increasingly vital… 25 The Information — AI news-outlet 1mo ago Defense Startup Shield AI in Talks for Valuation of at Least $20 Billion Shield AI, a startup building drones and AI-powered software for the military, is in talks to raise new funds at a valuation of at least $20 billion, according to a person with knowledge of the fundraise. The funding round, if it closes, would represent a roughly 60% increase to… 36 The Information — AI news-outlet 1mo ago Why China’s Answer to Surge AI Got a $1 Billion Valuation On Just $30 Million in Orders Unless you’ve been under a rock over the weekend, you’d know that the debate about AI safety took a big leap forward, as Anthropic CEO Dario Amodei on Saturday called for leading AI companies to slow the development of advanced AI, drawing supportive responses from OpenAI CEO… 22 Hugging Face Daily Papers research 1mo ago ActionSplice: In-Flight Action Editing for Interactive World Models Abstract ActionSplice introduces counterfactual state transport to splice revised actions into chunk-autoregressive video world models without replaying completed evaluations, improving fidelity and speed. Generated by thinkingmachines/Inkling-Small Chunk-autoregressive video… 38 Hugging Face Daily Papers research 1mo ago DataFlex-RL: An Evaluation Platform for RLVR Data Policies Abstract DataFlex-RL evaluates reinforcement learning data policies and finds that uniform sampling matches or exceeds adaptive rollout selection, reweighting, and domain mixing across math, logic, and science benchmarks. Generated by thinkingmachines/Inkling-Small Data policies… 8 arXiv — Machine Learning research 1mo ago Fundamental Dynamical Units for Physics-Informed Structural Inference from Perturbation Time-Series in Networked Systems arXiv:2609.11934v1 Announce Type: new Abstract: In networked dynamical systems, the parameter of primary mechanistic interest is signed interaction structure. Recovering this structure from perturbation time-series data is a fundamental identification problem, compounded by… 30 arXiv — Machine Learning research 1mo ago On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health arXiv:2609.11961v1 Announce Type: new Abstract: Stress is a pervasive determinant of mental health and a key target for mobile health interventions. On-device language models (ODLMs) offer privacy-preserving inference without cloud dependency, yet their feasibility for health… 35 arXiv — Machine Learning research 1mo ago Can We Trust LLM Judges: A Study of Capability-Dependent Biases and Multi-Judge Ensemble for Bias Calibration arXiv:2609.12002v1 Announce Type: new Abstract: LLMs are increasingly used as automated judges for model training and evaluation, yet individual judges exhibit systematic biases that undermine reliability. Much of prior work has studied biases in pairwise LLM-as-a-judge… 15 arXiv — Machine Learning research 1mo ago Certifying Concept Unlearning in Text-to-Image Diffusion Models arXiv:2609.12163v1 Announce Type: new Abstract: Existing evaluations of concept unlearning in text-to-image (T2I) diffusion models primarily rely on attack success rates obtained through automated adversarial prompt search. However, these metrics provide only empirical evidence… 16 arXiv — Machine Learning research 1mo ago PLSP (Pre-hoc Liminal Space Profiling): OOD Prediction over Detection -- An Anticipatory Approach for Machine Learning Model Reliability arXiv:2609.12225v1 Announce Type: new Abstract: Out-of-Distribution (OOD) data poses a significant threat to machine learning models, often leading to model failure during deployment. All existing OOD detection methods are post-hoc, relying on evaluation metrics such as accuracy… 5 arXiv — Machine Learning research 1mo ago ProactiveBench: Can Streaming Video Models Really Interact Like Humans? arXiv:2609.12658v1 Announce Type: new Abstract: Streaming video understanding requires models to process continuous multimodal input while maintaining temporal context. Existing evaluations are predominantly reactive: they query a model at a selected timestamp and therefore do… 26 arXiv — Machine Learning research 1mo ago Robust Policy Optimization via Adversarial Importance Sampling arXiv:2609.13044v1 Announce Type: new Abstract: Significant progress has been made in safeguarding deep reinforcement learning (DRL) policies against input perturbations. Developing robust DRL involves three main stages: algorithm design, implementation, and evaluation. In this… 14 arXiv — NLP / Computation & Language research 1mo ago GAUGE: When Not to Trust LLM-as-a-Judge in User-Simulated Evaluation of Task-Oriented Agents arXiv:2609.12191v1 Announce Type: new Abstract: Comparing and selecting task-oriented LLM agents increasingly relies on a low-cost offline evaluation gate: persona-driven LLM user-simulators converse with each candidate, an LLM-as-a-judge scores the transcripts, and the… 30 arXiv — NLP / Computation & Language research 1mo ago Zipbench: Low-Cost Framework for Compressing Comprehensive Benchmarks of Large Language Models arXiv:2609.12475v1 Announce Type: new Abstract: Comprehensive benchmark suites are essential for improving large language models (LLMs), but many widely used benchmarks are redundant, making evaluation unnecessarily expensive. Although recent benchmark compression methods (BCMs)… 13 arXiv — NLP / Computation & Language research 1mo ago Continue, Adapt, or Yield: In-Turn Adaptation to Overlapping Speech in Full-Duplex Agents arXiv:2609.13117v1 Announce Type: new Abstract: Full-duplex evaluation often emphasizes whether an agent keeps speaking or stops. That binary cannot express a third response humans use routinely: continuing to speak while incorporating what the listener just contributed. The… 38 arXiv — NLP / Computation & Language research 1mo ago Do LLMs Trust the Accuser or the Accusation? Measuring Belief Shifts in Werewolf arXiv:2609.12446v1 Announce Type: cross Abstract: Social-deduction games such as Werewolf are increasingly used to evaluate LLM agents, but existing evaluations often rely on final game outcomes. We propose a belief-shift evaluation benchmark in Werewolf for analyzing… 17 arXiv — NLP / Computation & Language research 1mo ago MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation arXiv:2512.14691v3 Announce Type: replace Abstract: Modern multimodal generative models can synthesize visually compelling images and videos, but it remains unclear whether this visual fluency reflects genuine reasoning: when prompted to generate a solution, can a model preserve… 8 arXiv — NLP / Computation & Language research 1mo ago From Bench-to-Bedside: A Review of Clinical Trials in Drug Discovery and Development arXiv:2412.09378v4 Announce Type: replace-cross Abstract: Clinical trials bridge basic research and clinical application, serving as essential steps in drug development. This review examines clinical trial phases (Phase I [safety assessment], Phase II [efficacy evaluation],… 38 Hugging Face Daily Papers research 1mo ago Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation Abstract Benchmark Radar is a searchable living database and discovery engine for AI evaluation benchmarks that aggregates sources, score histories, and evidence to support benchmark selection and comparison. Generated by thinkingmachines/Inkling-Small Benchmark researchers and… 7 Hugging Face Daily Papers research 1mo ago COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization Abstract COBRA-Skills improves LLM agent skill optimization by using contextual-bandit prioritization and evidence-based evolution to cut evaluation costs while maintaining high performance. Generated by thinkingmachines/Inkling-Small Large language model (LLM) agents can… 21 The Information — AI news-outlet 1mo ago Nvidia May Invest Up to $10 Billion In Anthropic’s IPO Nvidia has discussed investing in Anthropic’s upcoming initial public offering that would raise as much as $100 billion at a valuation of around $2 trillion, Reuters reported Friday. Nvidia could invest as much as $10 billion in Anthropic at the IPO price, according to the… 19 TechCrunch — AI news-outlet 1mo ago Mecka AI nears $500M valuation in Sequoia-led deal amid rush for robot training data The round for the two-year-old startup is coming together months after Mecka announced its Series A. 32 arXiv — Machine Learning research 1mo ago Counterfactual Marginalisation: Framework for Evaluating Robustness to Nuisance Variables arXiv:2609.10778v1 Announce Type: new Abstract: Machine learning models can achieve strong test performance while relying on demographic or acquisition-related shortcuts. We propose counterfactual (CF) marginalisation as a test-time evaluation procedure for assessing robustness… 25 arXiv — Machine Learning research 1mo ago When does a spectral prior help graph learning? Connectivity-loss estimation under road-network disruptions arXiv:2609.11166v1 Announce Type: new Abstract: Rapid evaluation of many simultaneous road-link disruptions requires a practical compromise between exact spectral recomputation and local approximation. We estimate relative algebraic-connectivity loss after multi-edge deletion… 22 arXiv — Machine Learning research 1mo ago CausalArena: Benchmarking Causal Discovery in the Foundation Model Era arXiv:2609.11897v1 Announce Type: new Abstract: Causal discovery aims to uncover causal structures from data and is fundamental to scientific reasoning and intervention-based decision making. Its evaluation relies heavily on structural causal models (SCMs), which specify a… 14 arXiv — Machine Learning research 1mo ago A Station-Based Evaluation of Machine Learning-based Weather Forecasting Models in Northern Norway arXiv:2609.10564v1 Announce Type: cross Abstract: Recent machine learning weather prediction (MLWP) models have demonstrated remarkable forecasting skill on global reanalysis-based benchmarks. However, their performance remains unclear in challenging environments such as… 27 arXiv — Machine Learning research 1mo ago An Empirical Measurement of Jailbreaking Evaluators arXiv:2609.10594v1 Announce Type: cross Abstract: Expert evaluation of jailbreak responses is costly and difficult to scale, so the community increasingly relies on automated evaluators to determine whether an attack succeeds. However, jailbreak studies typically validate their… 38 arXiv — Machine Learning research 1mo ago Sequence-Informed Geometric Evaluation of RNA 3D Structures arXiv:2609.10644v1 Announce Type: cross Abstract: Computational RNA structure pipelines generate many candidate conformations for the same sequence. Reliable evaluation therefore requires more than recognising plausible geometry, it requires determining whether that geometry is… 37 arXiv — NLP / Computation & Language research 1mo ago Rubric-Aligned Disentangled Evaluation of Human Simultaneous Interpreting arXiv:2609.11131v1 Announce Type: new Abstract: Human simultaneous interpreting (SI) is commonly assessed with analytic rubrics separating meaning transfer, delivery quality, and temporal synchrony, yet no automatic metric is designed for rubric-aligned segment-level SI… 5 arXiv — NLP / Computation & Language research 1mo ago Complex-Text Robustness Evaluation and Failure Diagnosis for Low-Resource Multilingual Text-to-Speech arXiv:2609.11545v1 Announce Type: new Abstract: Low-resource multilingual text-to-speech (TTS) systems have expanded language coverage, but their robustness under complex text inputs remains insufficiently diagnosed. Existing evaluations mainly focus on naturalness, speaker… 37 arXiv — NLP / Computation & Language research 1mo ago RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety arXiv:2609.11758v1 Announce Type: new Abstract: Allowing large language models (LLMs) to retrieve information from a set of trusted documents can increase reliability and reduce hallucination. However, recent work has demonstrated that retrieval-augmented generation (RAG) can… 25 arXiv — NLP / Computation & Language research 1mo ago The widening evaluation gap in medical large language model research 2023 to 2026 arXiv:2609.11770v1 Announce Type: new Abstract: Large language models are superseded every few quarters; clinical evidence takes years. We asked whether medical research is keeping pace with the systems it evaluates. PubMed returned 11,628 records for January 2023 to June 2026… 13 arXiv — NLP / Computation & Language research 1mo ago Beyond Word Error Rate: A Switch Aware Evaluation of ASR and Audio Language Models on English Yoruba Code-Switched Speech arXiv:2609.11786v1 Announce Type: new Abstract: Automatic speech recognition (ASR) systems and audio language models (audio LMs) now report low error rates on monolingual benchmarks, but their behavior on code switched speech in low resource, diacritic rich languages remains… 33 arXiv — NLP / Computation & Language research 1mo ago Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2 arXiv:2609.10935v1 Announce Type: cross Abstract: Membership inference attacks (MIAs) try to determine whether a specific record was used to train a model, a privacy risk that matters in natural language processing (NLP), where training data can contain sensitive user text. This… 19 arXiv — NLP / Computation & Language research 1mo ago Same Day, Same Story; One Day Ahead, a Different Signal: The Dual Validity of Financial Sentiment arXiv:2609.11144v1 Announce Type: cross Abstract: Financial NLP has a standard workflow: validate a sentiment tool against human labels, then trust it to extract market signal. This assumes the two evaluations measure the same thing. We test that assumption in a setting where… 6 arXiv — NLP / Computation & Language research 1mo ago "Mirror" Large Language Model Evaluations of Depression are Criterion Contaminated arXiv:2508.05830v3 Announce Type: replace Abstract: Large Language Model (LLM) studies that use language responses elicited from depression assessments to predict scores on those same assessments often report near-perfect prediction of depression. We refer to these as "Mirror"… 8 Page 7 of 10 · 500 articles ← Newer Older →
