News / #funding Tag Funding 500 articles archived under #funding · RSS Sign in to follow Hugging Face Daily Papers research 1mo ago On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability Abstract Qwen3.8-Flash-Next is a sparse mixture-of-experts architecture combining hybrid gated delta-net and sparse attention layers, gated residual branches, and off-accelerator n-gram embeddings to improve efficiency, capability, and training stability. Generated by… 5 arXiv — Machine Learning research 1mo ago Off-Policy Evaluation for Semantic ID Recommenders: Does the Model's Own Code Hierarchy Help? arXiv:2608.28905v1 Announce Type: new Abstract: Generative recommenders increasingly emit semantic IDs (SIDs): each item is a short sequence of hierarchical discrete codes from a residual quantizer, decoded autoregressively. Before spending scarce A/B-test, a team may decide… 34 arXiv — Machine Learning research 1mo ago Hybrid Semantic Context-Enhanced Ensemble Learning for Wind Power Ramp-Event Forecasting and Uncertainty-Aware Evaluation arXiv:2608.29024v1 Announce Type: new Abstract: Wind power ramp events which are sudden, large swings in turbine output over short windows are difficult to estimate, and standard models often miss them. Hybrid forecasting approach is built which augments semantic context to… 28 arXiv — Machine Learning research 1mo ago Scalable Clinical Data Infrastructure and Comparative ML Evaluation for Hospitalisation Risk Prediction in Elderly Patients with Multiple Long-Term Conditions using CPRD arXiv:2608.29419v1 Announce Type: new Abstract: Deep learning architectures are increasingly proposed for patient trajectory modeling in electronic health records (EHRs), yet their advantage over simpler, more interpretable models is rarely subjected to rigorous empirical… 25 arXiv — Machine Learning research 1mo ago One Capability or Many? Testing the Economic Validity of Frontier AI Evaluation arXiv:2608.29420v1 Announce Type: new Abstract: Frontier-model leaderboards now rank systems based on economic benchmarks, tests of how well models carry out professional tasks from software engineering to banking workflows, and those rankings inform what organisations buy, what… 4 arXiv — Machine Learning research 1mo ago Reference-Grafting Matches Fine-Tuning at Eliciting Sandbagged Capabilities arXiv:2608.29458v1 Announce Type: new Abstract: Sandbagging, in which a model deliberately underperforms on an evaluation despite retaining the underlying capability, threatens the safety evaluations that frontier-model governance depends on. The Elicitation Game found that… 14 arXiv — Machine Learning research 1mo ago A Causal Model for Locating and Unlocking Sandbagging in Model Organisms arXiv:2608.29461v1 Announce Type: new Abstract: Sandbagging models strategically underperform on evaluations while retaining the capabilities being measured. The evaluations that guide frontier-model deployment and governance then understate what these models can do. To… 10 arXiv — Machine Learning research 1mo ago Which LLM for Which Work? Budgeted Model Allocation under Uncertain Evaluation arXiv:2608.29560v1 Announce Type: new Abstract: A company with a fixed artificial intelligence (AI) budget must decide which large language model (LLM) handles each recurring workload. What it lacks is the quality table, how well each model performs on each workload. Given that… 7 arXiv — Machine Learning research 1mo ago Adaptive Doubly Robust Off-Policy Evaluation for Ranking Policies under Diverse User Behavior arXiv:2608.29600v1 Announce Type: new Abstract: Off-policy evaluation (OPE) of ranking policies is challenging be- cause selecting and ordering multiple items from a candidate set makes the number of possible rankings grow combinatorially with the number of candidates and the… 7 arXiv — Machine Learning research 1mo ago PruneShift: A Framework for Evaluating Decision Reliability in Structured Pruning arXiv:2608.29765v1 Announce Type: new Abstract: Structured pruning uses surrogate objectives because direct task evaluation over every feasible mask is too expensive. Most evaluations report average surrogate error or rank correlation on broadly sampled masks. These summaries do… 15 arXiv — NLP / Computation & Language research 1mo ago Do large language models scrutinise what they review? A multimodal audit of scoring calibration, error detection, and author-identity effects arXiv:2608.28626v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to generate peer reviews, prompting examination of their capacity for critical evaluation. This study evaluates two multimodal LLMs, Qwen2.5-VL-72B and Pixtral-Large-124B, as… 38 arXiv — NLP / Computation & Language research 1mo ago Terminal-Bench-LILT: Multilingual Agentic Coding Benchmark Grounded in Language, Region, and Culture arXiv:2608.28641v1 Announce Type: new Abstract: Most evaluations for coding agents are conducted exclusively in English, which does not reflect real-world multilingual deployment. We present Terminal-Bench-LILT, a suite of 300 authentic coding tasks in ten languages: Arabic,… 16 arXiv — NLP / Computation & Language research 1mo ago Redesigning and Auditing Deep Research Writing for Faithful Reports arXiv:2608.28643v1 Announce Type: new Abstract: Rubric-based evaluations of deep-research (DR) systems often obscure fine-grained factual failures in generated reports. We introduce CLAIMPROBE, a claim-level audit that decomposes DR reports into claims and measures… 28 arXiv — NLP / Computation & Language research 1mo ago Cross Lingual Transfer in Tulu Legal Comprehension: Script-Dependent Improvement and RAG-Induced Knowledge Conflict arXiv:2608.28645v1 Announce Type: new Abstract: Low-resource languages without an adequate training corpus often use a related, higher-resource language as a scaffold for comprehension. Still, there is a need to develop rigorous evaluation methods to identify when models fail in… 28 arXiv — NLP / Computation & Language research 1mo ago Toward Cultural Alignment: Human-Centered Evaluation of Multimodal AI Stories Across Five African Communities arXiv:2608.29209v1 Announce Type: new Abstract: In this paper, we examine how well AI-generated multimodal stories align with the lived practices, relationships, language, values, and visual expectations of the communities they represent. We conduct a community-grounded… 28 arXiv — NLP / Computation & Language research 1mo ago Large Language Models Systematically Favor Popular Options: Evidence and Mitigation Across MCQs arXiv:2608.29257v1 Announce Type: new Abstract: Multiple-choice questions (MCQs) are a standard format for evaluating large language models (LLMs), yet the popularity of answer options can confound evaluation. Modern LLMs systematically prefer popular but incorrect options over… 16 arXiv — NLP / Computation & Language research 1mo ago SHADOWBENCH: Toward Reliable Automatic Evaluation of Semantic Alignment in Autoformalization arXiv:2608.29270v1 Announce Type: new Abstract: Autoformalization translates informal mathematical theorems into code for proof assistants such as Lean. A central challenge is that current evaluation metrics can accept type-correct but misaligned statements or reject correct… 6 arXiv — NLP / Computation & Language research 1mo ago Evaluating the Semantic Specificity of Representation Steering in Language Models arXiv:2608.29431v1 Announce Type: new Abstract: Localized Representation Steering (LRS) is widely used to correct reasoning pathologies in large language models. However, standard benchmark evaluations can easily be fooled by superficial label overrides, creating a false… 5 arXiv — NLP / Computation & Language research 1mo ago Ontology-Guided Multi-Agent Extraction of Evaluation Objects from Academic Review Texts: Evidence from Chinese Library and Information Science arXiv:2608.29526v1 Announce Type: new Abstract: Academic reviews, scholarly commentaries, and book reviews serve as sources of evaluative statements about theories, methods, literature, institutions, and policies, providing valuable evidence for scholarly evaluation. Existing… 25 arXiv — NLP / Computation & Language research 1mo ago SUP-MIMIC: A Multi-Task Clinical Diagnosis Benchmark for Evaluating LLMs' Robustness to Contradictory Evidence arXiv:2608.29582v1 Announce Type: new Abstract: Current evaluations of large language models (LLMs) primarily focus on factual knowledge retrieval, overlooking the fundamental challenge of navigating the complex, non-bijective mappings between clinical indicators and diagnoses.… 12 arXiv — NLP / Computation & Language research 1mo ago DVBench: Benchmarking MLLMs for Understanding Dynamic Charts and Narratives in Data Videos arXiv:2608.29711v1 Announce Type: new Abstract: While MLLMs have made significant strides in chart comprehension and video understanding, current evaluations largely isolate these capabilities, leaving a critical gap in understanding temporally evolving structured visual… 29 arXiv — NLP / Computation & Language research 1mo ago GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluation arXiv:2608.29856v1 Announce Type: new Abstract: Large language models are increasingly used as scalable evaluators for open-ended tasks. However, many LLM judges derive query-specific criteria during scoring, leaving the evaluation requirements insufficiently specified and their… 26 Hugging Face Daily Papers research 1mo ago Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence Abstract This work proposes a structured ladder for scaling large reasoning models beyond human supervision by tracing autonomous rewards and self-generated experience, while identifying risks and evaluation dimensions. Generated by thinkingmachines/Inkling-Small Recent advances… 33 Hugging Face Daily Papers research 1mo ago PaperGym: Rubric-Centered Evolution for Research-Plan Generation Abstract PaperGym converts scientific papers into training environments by separating research questions from evaluation rubrics, enabling reinforcement learning that improves research planning across multiple model sizes. Generated by thinkingmachines/Inkling-Small Research… 22 The Information — AI news-outlet 1mo ago Andreessen Horowitz Raises Additional $1.75 Billion for Fifth Growth Fund Andreessen Horowitz said Monday it had raised an additional $1.75 billion for its fifth growth fund, bringing the total raised to $8.5 billion. Andreessen Horowitz raised $6.75 billion for that fund back in January. The firm said it raised more cash for the fund, which had… 6 TechCrunch — AI news-outlet 1mo ago Harvard Law dropout raises $6M for Blue Voice to build a ‘Harvey for police officers’ The seed round for the app that provides real-time legal and policy guidance to officers was led by SignalFire and Las Olas VC. 5 Hugging Face Daily Papers research 1mo ago Agentic Artifact Creation: Systems, Evaluation, Principles, and Opportunities Abstract This survey defines agentic artifact creation as stateful, feedback-driven construction of deliverables by AI systems and analyzes 259 works across artifact families, settings, and evaluation practices to propose principles for accountable control. Generated by… 35 Hugging Face Daily Papers research 1mo ago Paint What You See: Benchmarking Dexterous Visual Tool Use in Multimodal Agents Abstract EASEL benchmarks fine-grained visual tool use through reference-guided reconstruction and semantic tasks, revealing that current multimodal agents struggle with closed-loop precision and trajectory stability. Generated by thinkingmachines/Inkling-Small Evaluation is… 23 arXiv — NLP / Computation & Language research 1mo ago SciReC: Diagnostic Evaluation of Multimodal, Multi-Turn Relational Reasoning with Adaptive Interaction arXiv:2608.27461v1 Announce Type: new Abstract: Relational reasoning requires the process of perceptual understanding, comparing, and integrating the underlying relationships between concepts. This ability consists of multiple categories, such as analogical, structural, and… 8 arXiv — Machine Learning research 1mo ago RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests arXiv:2608.27831v1 Announce Type: cross Abstract: Coding agents are now commonly evaluated on the SWE-bench family of benchmarks, whose tasks are built from curated GitHub issues--long, structured, and information-rich. Real user requests, however, are typically far shorter and… 12 arXiv — NLP / Computation & Language research 1mo ago EvoHarmBench: Breaking Content Moderation with Iterative Human-Like Evasion arXiv:2608.27844v1 Announce Type: new Abstract: Existing evaluations of harmful content detection rely predominantly on static benchmarks, which struggle to reflect the interactive adversarial ecosystem of real-world content platforms where users continuously revise their… 36 arXiv — NLP / Computation & Language research 1mo ago CultureConverse: A Multilingual Multi-turn Simulation Harness for Culturally Grounded Assistance in East and Southeast Asia arXiv:2608.28405v1 Announce Type: new Abstract: Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking practical help over multiple turns in culturally… 26 arXiv — NLP / Computation & Language research 1mo ago Stranger, Fan, or Peer? A Systematic Study on the Role of Interlocutor in Persona-Based Dialogue Generation arXiv:2608.28467v1 Announce Type: new Abstract: Persona-based dialogue systems are usually conditioned on speaker biography, but dialogues involve at least two participants, and who has access to whose biography can vary across training, inference, and evaluation. Prior work… 22 arXiv — NLP / Computation & Language research 1mo ago Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluation arXiv:2608.28508v1 Announce Type: new Abstract: Forced alignment evaluation typically requires manually annotated timestamps, limiting large-scale and multilingual analysis. We introduce two corpus-level metrics based on self-supervised (SSL) speech representations for… 6 arXiv — NLP / Computation & Language research 1mo ago SURE-Challenge: Evaluating Speech Evidence Before Speech-LLM Generation arXiv:2608.27783v1 Announce Type: cross Abstract: Speech LLMs are usually graded after they answer, although an operating system first has to decide whether a waveform should be sent to the model. We define the Speech-Unsupported Rejection Evaluation Challenge (SURE-Challenge)… 33 arXiv — NLP / Computation & Language research 1mo ago Auditing Generative Audio Calls for Known-Task Audio-LLM Evaluation arXiv:2608.27817v1 Announce Type: cross Abstract: Speech and audio LLMs are often evaluated by asking whether a waveform prompt beats an automatic speech recognition (ASR) transcript. For known closed-set tasks, that comparison conflates two factors: access to acoustic evidence… 33 Hugging Face Daily Papers research 1mo ago LMSM: LLM Security Framework Inspired by Linux Security Modules Abstract LMSM applies a modular security framework to LLM serving by separating evidence calibration, policy evaluation, and output gating, enabling flexible interpretability-based enforcement without rebuilding request handling. Generated by thinkingmachines/Inkling-Small Large… 19 r/LocalLLaMA community 1mo ago We used HFlow to evaluate the latest open weights VLMs for processing egocentric data We used HFlow to evaluate the latest open weights VLMs for processing egocentric data. This was based on Build AI's Egocentric-10k evaluation , which used Gemini 2.5 Flash to measure hand visibility and active manipulation. We kept the same prompts and the same dataset, only… 17 r/MachineLearning community 1mo ago You can beat SOTA Time Series Anomaly Detection methods with a 100 year old algorithm [R] You can beat SOTA Time Series Anomaly Detection methods with a 100 year old algorithm Time Series Anomaly Detection (TSAD) seems to be one of the hottest topics in NeurIPS, SIGKDD, VLDB etc. Many (perhaps most) papers evaluate on Paparrizos’ TSB-AD-M benchmark… However, I tested… 4 The Information — AI news-outlet 1mo ago Andreessen Horowitz Raises $1.1 Billion for AI Hardware Fund Andreessen Horowitz raised $1.1 billion for a fund that will invest in physical AI and infrastructure startups, the firm announced Friday. The money will be used to back companies building chips, memory, networking, storage and other companies involved in the AI buildout.… 15 TechCrunch — AI news-outlet 1mo ago Neocloud Lambda secures $1B in debt to buy more chips Neocloud Lambda has raised $1B in private debt to buy Nvidia AI chips and lease them to Microsoft. It's the latest in a string of loans, underscoring the high cost of the AI boom. 17 Hugging Face Daily Papers research 1mo ago What Does an Evaluation License? A Commit-Bound Census of Claim-Relative Inference in Inspect Evals Abstract Evaluation artifacts specify a forward computation: a task, scorer, and reported metric. They do not necessarily license the claim attached to that metric because the historical evidence and alternative semantics needed to replay it may be unbound. We formalize this… 25 arXiv — Machine Learning research 1mo ago NeuronFuzz: Safety Neuron Guided Fuzzing for LLM Safety Evaluation arXiv:2608.26222v1 Announce Type: new Abstract: Safety evaluation is critical for assessing whether aligned Large Language Models (LLMs) remain robust against jailbreak attacks. Existing automated testing methods, however, largely rely on response-level feedback: each candidate… 12 arXiv — Machine Learning research 1mo ago Tabular Deep Learning for Algorithmic Trading: Cross-Regime Bayesian Optimisation for Equity Signal Generation arXiv:2608.27076v1 Announce Type: new Abstract: Algorithmic trading now represents a market exceeding $20 billion, where even marginal gains in signal robustness can translate into economically significant returns. Existing evaluations of equity prediction models do not… 24 arXiv — Machine Learning research 1mo ago TraceBench: Controlled Evaluation of LLM Agents for Time-Series Root-Cause Attribution arXiv:2608.27182v1 Announce Type: new Abstract: LLM agents are increasingly applied to anomaly detection and root-cause analysis in time-series observations collected from real-world systems; however, their performance on these tasks has not been systematically evaluated under… 10 arXiv — Machine Learning research 1mo ago Profit based evaluation of machine learning for nitrogen recommendations in winter wheat arXiv:2608.27205v1 Announce Type: new Abstract: Nitrogen rates for winter wheat are set before the season, under unknown prices and weather. The standard UK advice does not respond to prices, yet recent price swings moved the most profitable rate by tens of kilograms per… 26 arXiv — NLP / Computation & Language research 1mo ago Cross-Platform Generalisation Failure in Mental Health Natural Language Processing: A Five-Axis Fairness Audit of Transformer Models on Social Media arXiv:2608.26138v1 Announce Type: new Abstract: We introduce the Cross-Platform Fairness Evaluation (CPFE) framework -- a five-axis audit protocol covering discriminative performance, calibration, statistical significance, prediction equity, and attribution stability -- and… 34 arXiv — NLP / Computation & Language research 1mo ago ElementCheck: Complexity-Aware Long-Form Text Factuality Evaluation via Sentence Elements arXiv:2608.26118v1 Announce Type: new Abstract: Existing long-form factuality evaluation relies on the decompose-retrieve-verify pipeline. However, the pipeline suffers from noise from claim decomposition and fixed verification granularity, resulting in unreliable results. We… 26 arXiv — NLP / Computation & Language research 1mo ago Evaluating Language Models in Realistic Conversational Contexts arXiv:2608.26131v1 Announce Type: new Abstract: As Large Language Models (LLMs) are increasingly deployed to serve open-ended, multi-turn interactions, evaluating conversational quality at human scale has become a central challenge. Existing evaluation frameworks built for… 20 arXiv — NLP / Computation & Language research 1mo ago Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation arXiv:2608.26159v1 Announce Type: new Abstract: Self-Generated Text Recognition (SGTR)--the ability of an LLM to identify its own outputs--poses risks to AI safeguards that rely on LLMs as evaluators or monitors. Specifically, an LLM may recognize outputs from other copies of… 34 Page 10 of 10 · 500 articles ← Newer
