News / #funding Tag Funding 500 articles archived under #funding · RSS Sign in to follow arXiv — Machine Learning research 1mo ago Beyond Endpoint Scores: Time- and Capacity-Conditioned Evaluation of Continual Knowledge Updating arXiv:2609.03900v1 Announce Type: new Abstract: Continual knowledge-updating methods are often declared superior from one final checkpoint and one conventional adapter rank. We show that this can be insufficient to identify the better operating point. Holding a periodic… 27 arXiv — Machine Learning research 1mo ago You Can't Escape Your Own Activations : Evaluation Awareness and Multi-Agent Monitoring arXiv:2609.03035v1 Announce Type: cross Abstract: LLM agents are increasingly deployed in multi-agent systems, where they can collude while keeping their actions benign. Output monitors designed to detect such collusions can be fooled by obfuscation and steganography, motivating… 38 arXiv — NLP / Computation & Language research 1mo ago Judging LLM-as-a-Judge: Concerning Rubric Artifacts in LLM-based Automated Text Generation Evaluation arXiv:2609.02942v1 Announce Type: new Abstract: LLM-as-a-Judge pipelines are increasingly used to evaluate AI-generated text, based on the assumption that judgments arise from reasoning over candidate responses with respect to a rubric. We show that this assumption warrants… 12 arXiv — NLP / Computation & Language research 1mo ago FPCO-Dialog: A Multi-Turn False-Premise Benchmark for Correction and Cooperation in Vision-Language Models arXiv:2609.03331v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly deployed in multi-turn settings where users may describe visual content with incorrect assumptions. Yet existing evaluations rarely isolate how models respond when the same visually… 19 arXiv — NLP / Computation & Language research 1mo ago Decoupled Analysis-Judging: An Automated Creativity Evaluator Using LLMs in Complex Multi-step Creativity Tasks arXiv:2609.03432v1 Announce Type: new Abstract: Automated evaluation of creativity tasks remains challenging for LLM-as-a-Judge, as LLM is susceptible to biases such as verbosity bias and leniency bias. Such limitations are particularly evident in Contextually-Grounded and… 22 arXiv — NLP / Computation & Language research 1mo ago Last Translation Benchmark arXiv:2609.04173v1 Announce Type: new Abstract: For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, standard benchmarks for machine translation are… 20 arXiv — NLP / Computation & Language research 1mo ago VoxReason: Listener-Free Evaluation of Source-Grounded Speech Planning Before Synthesis arXiv:2609.03203v1 Announce Type: cross Abstract: Expressive speech systems make a decision before any waveform is rendered: how an utterance is delivered. In dialogue agents, narration, and role-conditioned TTS, that hidden planning step sets affect, pitch, energy, rate, pause,… 16 TechCrunch — AI news-outlet 1mo ago Crusoe reportedly raises $3B at a $30B valuation The round came together after the data center developer reportedly secured a $13 billion contract with Jane Street. 32 TechCrunch — AI news-outlet 1mo ago Accel reportedly in talks to lead $1B round for Thinking Machines at $40B valuation The high-profile startup's annual revenue run rate stands at at over $100 million. 28 The Information — AI news-outlet 1mo ago Thinking Machines Lab In Talks to Raise Billions at Roughly $40 Billion Valuation Thinking Machines Lab, the AI developer led by ex-OpenAI chief technology officer Mira Murati, is in talks to raise at least $1 billion at a valuation of at least $40 billion before the investment, according to a person with knowledge of the fundraise. Existing investor Accel is… 28 arXiv — Machine Learning research 1mo ago GeoSPRINT: Geometric Redundancy-Aware Step Pruning for Inference in Diffusion Trajectories arXiv:2609.02160v1 Announce Type: new Abstract: Diffusion models achieve high sample quality but remain expensive at inference time because sampling requires many sequential neural function evaluations (NFEs). Existing acceleration methods either use fixed step-skipping… 20 arXiv — Machine Learning research 1mo ago Bayes-Optimal BER and AUC: Estimation and Evaluation of Estimators arXiv:2609.02304v1 Announce Type: new Abstract: A fundamental quantity in machine learning is the optimal performance achievable by any model on a given task. Estimating this quantity allows us to distinguish the irreducible part of the error from a deficiency of the model,… 30 arXiv — Machine Learning research 1mo ago Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM Weights arXiv:2609.02652v1 Announce Type: new Abstract: Leech-lattice vector quantization holds the strongest reported 2-bit quality under its own evaluation protocol. Its kernel decodes one shell; we found no implementation of the multi-shell decoder the rate requires. This paper… 30 arXiv — Machine Learning research 1mo ago Hybrid Retrieval-Augmented Generation with Knowledge Graph Expansion, RRF Fusion, and Per-Chunk Grounded Evaluation for Enterprise Document Search arXiv:2609.01617v1 Announce Type: cross Abstract: Getting accurate, grounded answers out of large enterprise document repositories is a difficult problem. Dense vector retrieval alone frequently performs poorly on queries that mix technical terminology, vendor-specific acronyms,… 18 arXiv — Machine Learning research 1mo ago FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making arXiv:2609.01691v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used to make decisions from visual inputs. We introduce FAIRLENS, a benchmark and evaluation framework for measuring both the fairness and the validity of VLM responses in three… 21 arXiv — Machine Learning research 1mo ago Ten Architectures, One Error: Shared Failure Modes in Hyperspectral Classification under Spatially Disjoint Evaluation arXiv:2609.01786v1 Announce Type: cross Abstract: Hyperspectral image classification still relies heavily on random pixel splits within a single scene. The Salinas dataset, randomly split, is among the most widely used datasets for comparing different architectures. However,… 17 arXiv — NLP / Computation & Language research 1mo ago VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages arXiv:2609.01788v1 Announce Type: new Abstract: Real-world communication often requires pragmatic reasoning: interpreting meanings implied through context and cultural convention rather than stated literally. Existing pragmatic evaluation remains largely limited to English and… 18 arXiv — NLP / Computation & Language research 1mo ago Do Cantonese-Adapted Language Models Better Predict Cantonese Reading? A Cross-Model Eye-Tracking Evaluation arXiv:2609.02163v1 Announce Type: new Abstract: Information-theoretic measures derived from autoregressive language models are widely used to characterize the expectations that shape human reading, but whether language-variety-specific training improves such psycholinguistic… 30 arXiv — NLP / Computation & Language research 1mo ago Improving Health Literacy through Lay Summarization of Radiological Reports: An Evaluation of BioNER and Retrieval-Augmented Generation arXiv:2609.02396v1 Announce Type: new Abstract: Radiology reports are written primarily for clinicians, and their specialized terminology often makes them difficult for patients to interpret. As a result, many patients turn to publicly available Large Language Models (LLMs) to… 26 arXiv — NLP / Computation & Language research 1mo ago Before the Script, Set the Stage: How Worldview Simulation Amplifies Psychologically Grounded Persuasion in Multi-Turn Jailbreaking arXiv:2609.02414v1 Announce Type: new Abstract: Multi-turn jailbreak attacks demonstrate that harmful intent can be distributed across dialogue, yet existing methods obscure what conversational mechanisms drive vulnerability. We introduce BLUEPRINT, a safety-evaluation framework… 37 arXiv — NLP / Computation & Language research 1mo ago EarlyEval: Cheaper Agent Evaluation via Early Outcome Prediction arXiv:2609.02783v1 Announce Type: new Abstract: Evaluating LLM agents is essential for guiding their development, yet it has grown prohibitively expensive: a single pass of a frontier model over an agentic benchmark can cost hundreds to thousands of dollars, a price paid… 24 arXiv — NLP / Computation & Language research 1mo ago EvalDetectBench: A Benchmark for Measuring Evaluation Awareness in Frontier Language Models arXiv:2609.01611v1 Announce Type: cross Abstract: Frontier large language models can often recognize when they are being evaluated, a capability known as evaluation awareness. If models behave differently in evaluations than in deployment, this undermines the validity of… 37 arXiv — NLP / Computation & Language research 1mo ago Accurate in space, unreliable in time: how LLMs represent national cultural change arXiv:2609.01902v1 Announce Type: cross Abstract: Assessments of cultural alignment have become an important part of the development and improvement of large language models (LLMs). However, the majority of the evaluations treat culture as a single snapshot, investigating only… 15 arXiv — NLP / Computation & Language research 1mo ago Improving Evaluation Realism with Inference-Time Compute and Deployment Scaffolds arXiv:2609.02302v1 Announce Type: cross Abstract: A core obstacle to alignment evaluation is evaluation awareness: capable models can tell when they are being tested rather than deployed, weakening the conclusions a safety evaluation can support. We present two techniques that… 8 arXiv — NLP / Computation & Language research 1mo ago Incremental Pooled LLM Evaluation for Cost-Effective Retrieval Model Selection arXiv:2609.02745v1 Announce Type: cross Abstract: Selecting a retrieval model for a production RAG system requires reliable comparative evaluation, but obtaining relevance judgments at scale is expensive and difficult to repeat as new candidate systems arrive. We study pooled… 33 Hugging Face Daily Papers research 1mo ago EarlyEval: Cheaper Agent Evaluation via Early Outcome Prediction Abstract EarlyEval predicts agent outcomes from intermediate behavior to reduce evaluation cost by halting runs early with minimal accuracy loss. Generated by thinkingmachines/Inkling-Small Evaluating LLM agents is essential for guiding their development, yet it has grown… 37 Hugging Face Daily Papers research 1mo ago RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests Abstract Real-world coding requests are shorter and more casual than benchmark tasks, and explicitly stating desired behavior and motivation improves LLM software engineering performance. Generated by thinkingmachines/Inkling-Small Coding agents are now commonly evaluated on the… 9 TechCrunch — AI news-outlet 1mo ago Wonderful more than doubles its valuation to $5B in under 6 months Wonderful said it will use its $550 million Series C funding to develop products faster, expand its FDE teams, and meet demand for its products. 4 TechCrunch — AI news-outlet 1mo ago HiddenLayer nabs $100M as enterprises rush to secure their AI deployments HiddenLayer has raised a $100M Series B from Delta-v Capital, Ten Eleven Ventures, Morgan Stanley, Microsoft's M12, Booz Allen Hamilton, and others. 38 arXiv — Machine Learning research 1mo ago Local Reference Geometry Residual Augmentation for Imbalanced Time Series Classification arXiv:2609.00093v1 Announce Type: new Abstract: Imbalanced time series classification is often addressed by changing the training distribution, objective, logits, or final threshold. These interventions address important biases, yet leave a representation-level question… 12 arXiv — Machine Learning research 1mo ago When Does Online Adaptation Pay on the Edge? A Leakage-Free Evaluation of Warmup, Learning-Rate Selection, and Resource Trade-offs for Time-Series Forecasting arXiv:2609.01126v1 Announce Type: new Abstract: Online adaptation can help edge time-series forecasting under distribution drift, but its measured benefit is sensitive to evaluation choices. We study six public multivariate streams, including building-sensor and smart-meter… 16 arXiv — Machine Learning research 1mo ago Solving In-Table Prediction Problems by Deep Neural Networks with Performance Evaluation Using Synthetic Data arXiv:2609.01262v1 Announce Type: new Abstract: Tabular deep learning (TDL) leverages neural networks (NN) to extract patterns from tabular data. Traditional TDL methods follow a supervised learning paradigm, where a target feature is explicitly given. In this work, however, we… 22 arXiv — Machine Learning research 1mo ago SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers arXiv:2609.01343v1 Announce Type: new Abstract: Looped Transformers increase effective depth by iterating a shared block of layers, but most evaluations compare at fixed model size, conflating architectural advantage with extra FLOPs. We study looping on Mixture-of-Experts… 22 arXiv — NLP / Computation & Language research 1mo ago trajectory-judge: What Outcome-Only LLM Judges Miss on Agent Trajectories arXiv:2609.00038v1 Announce Type: new Abstract: Outcome-only evaluation is the production default for LLM agents: show a judge the request and the final reply and ask whether it was handled well. The metric is structurally blind to an agent that reaches the right answer the… 17 arXiv — NLP / Computation & Language research 1mo ago RePro: Proof-Verified Benchmark Rewriting for Reliable Evaluation of LLM Mathematical Problem Solving arXiv:2609.00062v1 Announce Type: new Abstract: Data contamination undermines the reliable evaluation of large language models (LLMs) on mathematical problem solving. While rewriting-based evaluation mitigates memorization, existing methods lack guarantees of problem validity… 8 arXiv — NLP / Computation & Language research 1mo ago Synthetic Worlds for Temporal Evaluation and Knowledge Updating in LLMs arXiv:2609.00184v1 Announce Type: new Abstract: Large language models (LLMs) rely on static pretraining corpora, causing their knowledge to become outdated over time. Existing approaches for evaluating knowledge edits either suffer from rapid contamination or rely on… 14 arXiv — NLP / Computation & Language research 1mo ago NSIDDx: A Design Framework for Neuro-Symbolic, Practitioner-First Differential Diagnosis in Low-Resource Settings arXiv:2609.00256v1 Announce Type: new Abstract: LLM-based diagnostic systems achieve high semantic accuracy on benchmarks, but open-ended evaluation on clinically uncommon presentations reveals a systematic gap between headline accuracy and verifiable clinical reliability. We… 29 arXiv — NLP / Computation & Language research 1mo ago Toward Workflow-Aware Benchmarking for Healthcare NLP Agents arXiv:2609.00296v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly proposed for healthcare tasks such as clinical documentation, evidence retrieval, patient messaging, and care coordination. Yet many evaluations remain limited to static medical… 18 arXiv — NLP / Computation & Language research 1mo ago Human-Anchored Factuality Evaluation with Strategic Annotation arXiv:2609.00494v1 Announce Type: new Abstract: LLM-based factuality judges provide scalable evaluation signals, but their metrics are often systematically biased relative to human judgments. We study human-anchored factuality evaluation under limited annotation budgets, where… 9 arXiv — NLP / Computation & Language research 1mo ago Skill Following: Evaluating Actual Skill Use in Retrieval-Enabled LLM Agents arXiv:2609.00549v1 Announce Type: new Abstract: Large Language Model (LLM) agents increasingly rely on external skills, yet standard evaluations obscure whether retrieving these skills actually helps. Aggregate metrics often compare retrieved versus non-retrieved tasks,… 8 arXiv — NLP / Computation & Language research 1mo ago Investigating Assistant Bias in LLM User Simulators Using a Role Vector arXiv:2609.00608v1 Announce Type: new Abstract: LLM-based user simulators are increasingly used to evaluate autonomous agents at scale, in place of costly human evaluations. Despite this promise, these simulators exhibit "assistant bias," a tendency to cooperate and pursue task… 24 arXiv — NLP / Computation & Language research 1mo ago Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling arXiv:2609.00949v1 Announce Type: new Abstract: Multi-turn tool calling is a core evaluation scenario for large language model (LLM) agents. On public tool-calling benchmarks, open-weight models now approach or even surpass closed-source frontier models in aggregate accuracy.… 36 arXiv — NLP / Computation & Language research 1mo ago Disclosure-Gated User Simulation for Companion-Agent Evaluation arXiv:2609.00982v1 Announce Type: new Abstract: Using a large language model to play the user is now standard in scalable evaluation. It has a repeatedly diagnosed failure: the simulated user is excessively cooperative, so a system under test can score by the sheer number of… 26 arXiv — NLP / Computation & Language research 1mo ago Post-hoc Alignment of LLM-judges to Human Judgment Distribution arXiv:2609.01073v1 Announce Type: new Abstract: The LLM-as-a-judge (LLMaJ) framework offers a cost-effective and reproducible solution for automatic evaluation. However, current evaluation practices typically compare LLMaJ judgments against aggregated ground-truth labels,… 4 arXiv — NLP / Computation & Language research 1mo ago Does task decomposition improve automatic NLG evaluation? arXiv:2609.01139v1 Announce Type: new Abstract: The LLM-as-a-judge (LLMaJ) framework has emerged as a promising solution for cheap, reproducible, reference-free Natural Language Generation (NLG) evaluation. Prior work seeks to improve LLMaJ by decomposing evaluation tasks into… 19 arXiv — NLP / Computation & Language research 1mo ago Towards AI-Assisted Clinical Trial Matching: Practical Considerations, Multicenter Evaluation, and Real-World Deployment arXiv:2609.01202v1 Announce Type: new Abstract: Clinical trials are essential for advancing cancer care and drug development, but many fail because of insufficient patient enrollment. While there is growing interest in using AI to support patient recruitment, existing systems… 4 Hugging Face Daily Papers research 1mo ago InternReviewer & InternAdvocate: Objective Reward and Evaluation for Agentic Reinforcement Learning in Peer Review and Rebuttal Abstract Specialized scholarly agents use reinforcement learning and real-time citation verification to improve reasoning and factual accuracy in peer review and rebuttal generation. Generated by thinkingmachines/Inkling-Small Generating professional scholarly content, such as… 18 TechCrunch — AI news-outlet 1mo ago AfterQuery reportedly becomes Y Combinator’s fastest-ever unicorn, now valued at $3.2B AI model-training startup AfterQuery has reportedly raised a round that valued it at $3.2 billion, just five months after announcing its $30 million Series A at a $300 million valuation in April. 6 The Information — AI news-outlet 1mo ago Wafer, An Inference Provider That Uses Non-Nvidia Chips, Lands Acquisition Offers and $200 Million-Plus Valuation Developers and chip designers are increasingly upbeat about the benefits of using AI to supercharge the process of designing and optimizing AI chips, as my colleague Phoebe wrote about last week. Now, a year-old inference provider, Wafer , is trying a similar approach: using AI… 25 Hugging Face Daily Papers research 1mo ago ContextBias: Controlled Evaluation of Bias Persistence Under Context Shift in Text-to-Image Models Abstract Text-to-image models exhibit persistent role-linked visual stereotypes across varied contexts, with cross-role attribute concentration increasing rather than diminishing under unrelated prompts. Generated by thinkingmachines/Inkling-Small Text-to-image models learn… 13 Page 9 of 10 · 500 articles ← Newer Older →
