News / #funding Tag Funding 500 articles archived under #funding · RSS Sign in to follow r/MachineLearning community 1d ago Is human evaluation through surveys common in ML research?[R][D] Hi, I am a master's student working on a project where our fine-tuned AI model provides feedback on images. We are also comparing its feedback with outputs from Qwen/OpenAI APIs on the same images. My postdoc and PhD mentors suggested creating a Google Form where participants… 36 The Information — AI news-outlet 1d ago Applied Intuition Fields Offers at $30 Billion Valuation For the last three years, AI labs like OpenAI, Anthropic and xAI have sucked up a lot of investor attention—as well as nosebleed valuations. Now a growing number of startups that make apps, agents or other software powered by AI are fetching valuations of $20 billion or higher.… 32 r/LocalLLaMA community 2d ago Boole-1.7B: A 1-bit Qwen3-1.7B model that outperforms Bonsai-1.7B We're launching Precision Numerics by releasing Boole-1.7B, a 1-bit version of Qwen3-1.7B. 247 MB packed weights: 13.9× smaller than BF16. 85.9% of the BF16 model’s average score across our 10-task evaluation - outperforms 1-bit Bonsai-1.7B and nearly matches 1.58-bit… 10 arXiv — Machine Learning research 2d ago NV-Tesseract-AD: Curriculum-Enhanced Conditional Diffusion for Multivariate Time-Series Anomaly Detection arXiv:2610.12754v1 Announce Type: new Abstract: We propose NV-Tesseract-AD, a curriculum learning-enhanced conditional diffusion framework for multivariate time-series anomaly detection. Our central hypothesis is that reconstruction difficulty can be explicitly structured along… 38 arXiv — Machine Learning research 2d ago DataCupid: Stable Matching Repair for Training Data Acquisition with Delayed Feedback arXiv:2610.12960v1 Announce Type: new Abstract: Matching repair after utility changes can reuse valid observations, but new evaluations must respect matching capacity and may return out of order. We study announced invalidations with bounded, possibly reward-dependent reporting… 33 arXiv — Machine Learning research 2d ago Accelerating Linear Bandits with Persistent Candidate Sets arXiv:2610.13407v1 Announce Type: new Abstract: Exhaustive optimistic scoring can make action selection costly in linear bandits with a large fixed set of $K$ actions, requiring $\mathcal{O}(K)$ score evaluations per round. We introduce Persistent Candidate LinUCB… 23 arXiv — NLP / Computation & Language research 2d ago perturber: A Stateless Service for Semantics-Preserving Text Perturbations arXiv:2610.12667v1 Announce Type: new Abstract: Large language model (LLM) evaluations on clean benchmarks overstate model robustness: real inputs arrive perturbed, and these meaning-preserving variants routinely move model answers. Studying this systematically requires applying… 21 arXiv — NLP / Computation & Language research 2d ago GeoFitBench-LLM: A Benchmark for Assessing Large Language Models' Capabilities in Geospatial Data Requirements Elicitation for Geo-Analytic Tasks arXiv:2610.12669v1 Announce Type: new Abstract: Large language models are becoming central to GeoAI systems, yet their evaluation rarely considers the elicitation of geospatial data requirements. This omission is critical as retrieval oriented geo-analytic tasks cannot be… 19 arXiv — NLP / Computation & Language research 2d ago DistressBench: Evaluating Multi-Dimensional Crisis Support in Large Language Models arXiv:2610.12758v1 Announce Type: new Abstract: Language models have been adopted faster than evaluation practices that reliably assess their risks, and a growing share of use is affective rather than informational. Within that traffic users disclose suicide and self-harm (SSH)… 24 arXiv — NLP / Computation & Language research 2d ago People Making Conversation: Evaluating How Naturally Speech Models Converse arXiv:2610.12802v1 Announce Type: new Abstract: A conversational speech model has to decide, at every pause, whether to speak. Existing evaluations test boundary detection, policy adherence, or aggregate statistics, not this decision. We place real-time speech systems at 427… 6 arXiv — NLP / Computation & Language research 2d ago Personalization Without Stereotyping: Do Speech LLMs Know When to Turn a Deaf Ear? arXiv:2610.13550v1 Announce Type: new Abstract: Recent speech LLMs can leverage rich paralinguistic cues, enabling more complex and personalized interactions. However, little work investigates how this sensitivity can lead to stereotyping, and existing evaluations largely… 18 arXiv — NLP / Computation & Language research 2d ago AMLworld-Compact: Importance-Weighted Downsampling for Cost-Effective LLM Evaluation and Error Diagnosis arXiv:2610.13627v1 Announce Type: new Abstract: Evaluating large language models (LLMs) on AMLworld's smallest test splits is costly because each of over a million targets requires a separate graph prompt. Sampling reduces this cost, but small balanced subsets change unweighted… 13 arXiv — NLP / Computation & Language research 2d ago Retractions Revisited: A Replication, Extension, and Mechanistic Evaluation of Linguistic Obfuscation Effects in Fraudulent Science arXiv:2610.13678v1 Announce Type: new Abstract: Prior work suggests fraudulent scientists write their articles with higher rates of linguistic obfuscation than genuine scientists, with a possible mechanism suggesting obfuscation makes research more costly for a reader to… 30 arXiv — NLP / Computation & Language research 2d ago Not Just an Evaluation: A Case Study of Discourse Particle Sensitivity in Language Models arXiv:2610.13794v1 Announce Type: new Abstract: Discourse particles subtly shape the flow of discourse by reflecting conversational context and interlocutor attitudes, often leaving the truth-conditional meaning untouched. They are also often polyfunctional, with distinct senses… 34 r/LocalLLaMA community 4d ago Typesafe ai raised 870m $ on hype (jev) https://typesafe.ai/blog/series-ai Meanwhile one of the reddit members apparently developed the same architecture a year ago: https://www.reddit.com/r/LocalLLaMA/s/8zQXqCNDlb Looks like ai valuation now on the peak of hype. Who yelled louder got most of the invest and money.… 22 Latent.Space news-outlet 4d ago [AINews] TypeSafe/Jev at >$100M ARR, $7.5B valuation 3 weeks after launch Wow. 22 TechCrunch — AI news-outlet 4d ago Anthropic can’t reliably control its AI agents. It’s cutting off its internal evals from the live internet instead Anthropic said it "turned off live internet access" for "all our internal evaluations" until further notice. 37 TechCrunch — AI news-outlet 4d ago The maker of non-text AI model Jev valued at $7.5B just weeks after launch TypeSafe AI raised $870 million in a round led by a16Z. 24 Hacker News — AI on Front Page community 5d ago Our $445M Series D Article URL: https://oxide.computer/blog/our-445m-series-d Comments URL: https://news.ycombinator.com/item?id=50020014 Points: 279 # Comments: 95 34 The Information — AI news-outlet 5d ago Nvidia-Backed Firmus Scraps $5 Billion Australia IPO as Markets Wobble Firmus Grid, an AI cloud company backed by Nvidia, has pulled what would have been Australia’s biggest initial public offering in almost 30 years. The Sydney-based company had planned to raise nearly $5 billion at a valuation above $30 billion. It cited market volatility and… 31 arXiv — NLP / Computation & Language research 5d ago Conversational Task Disambiguation over Tabular Data: Leakage-Aware Formulation, Benchmark Suite, and Training arXiv:2610.10740v1 Announce Type: cross Abstract: Conversational task disambiguation over tabular data uses dialogue to resolve missing information about a user's intended task before producing a solution over tables or databases. Existing evaluation and training lack a… 18 arXiv — Machine Learning research 5d ago Evaluating Rubric Generation with Interventional Transfer arXiv:2610.10809v1 Announce Type: new Abstract: Instance-specific rubrics are common in AI benchmarks where reliable evaluation requires specific expert knowledge. This approach is difficult to scale, prompting research into the generation of rubrics with large language models… 4 arXiv — Machine Learning research 5d ago Amortized Off-Policy Evaluation for LLMs arXiv:2610.10848v1 Announce Type: new Abstract: Accurate evaluation is central to selecting which LLM to deploy, yet testing a candidate on live traffic exposes real users to an unvetted model. Teams therefore evaluate candidates offline, on data produced by already-deployed… 16 arXiv — Machine Learning research 5d ago RH-Detect: A Unified Benchmark for Reward Hacking Detection arXiv:2610.10947v1 Announce Type: new Abstract: Reward hacking, where a model exploits an evaluation signal without completing the intended task, threatens the reliability of deployed language model systems. Existing datasets use different labels, response formats, and metadata… 16 arXiv — Machine Learning research 5d ago Budgeted Multi-Source Counterfactual Annotation for Off-Policy Evaluation arXiv:2610.10974v1 Announce Type: new Abstract: Off-policy evaluation (OPE) estimates the value of a target policy from logged data, but limited behavior-policy coverage can force high-variance reweighting or reward-model extrapolation. Counterfactual annotations can add… 38 arXiv — Machine Learning research 5d ago RideBench: A Large-Scale Exogenous-Aware Benchmark for Ride-Hailing Time Series Forecasting arXiv:2610.11164v1 Announce Type: new Abstract: We release Ride-Hailing, a large-scale ride-hailing time series dataset synthesized from DiDi's marketplace data across 200 spatial areas. Ride-Hailing spans four consecutive years at half-hourly granularity and covers three… 36 arXiv — Machine Learning research 5d ago Low-Cost Sensor Calibration for Indoor Air Quality Monitoring: A Dataset, Evaluation Scenarios, and a Lightweight Model arXiv:2610.11236v1 Announce Type: new Abstract: Low-cost sensors enable scalable indoor air quality monitoring but require calibration because of nonlinear distortions, noise, and temporal drift. The conventional strict pairwise calibration setting requires a co-located… 36 arXiv — Machine Learning research 5d ago Bernoulli Flow Models: Self-Consistent Generative Modeling for Binary Data arXiv:2610.11362v1 Announce Type: new Abstract: Binary diffusion models typically require a large number of function evaluations (NFEs) to generate high-quality samples, making practical inference computationally expensive. Reducing NFEs while preserving sample quality without… 35 arXiv — NLP / Computation & Language research 5d ago Large Language Models for Machine Translation Quality Annotation: Humans and Models Are Both Challenged arXiv:2610.10918v1 Announce Type: new Abstract: Large Language Models (LLMs) are considered to be a more efficient and cost-effective alternative to human judgment for Machine Translation (MT) evaluation. With MT evaluation spanning a large number of language pairs, domains and… 17 arXiv — NLP / Computation & Language research 5d ago AI4Fire: Evaluating Large Language Models on Wildfire Tasks arXiv:2610.10946v1 Announce Type: new Abstract: Large language models (LLMs) are entering wildfire management, where overstated evaluations can cost property and lives. How do they perform on wildfire tasks, with and without grounding? Bare means a model receives the task input… 30 arXiv — NLP / Computation & Language research 5d ago Back in Style: A Sociolinguistic Approach to Authoring and Measuring Persona Fidelity in User Simulation arXiv:2610.10988v1 Announce Type: new Abstract: As agentic systems gain commercial popularity, user simulators increasingly serve as measurement instrument for their evaluation. However, the fidelity of simulated users in comparison to real human users is generally low, and… 24 arXiv — NLP / Computation & Language research 5d ago AdaptEvo: Adaptive Agent Learning with Evolving Supervision arXiv:2610.11354v1 Announce Type: new Abstract: Rule-governed contextual decision tasks require models to apply specified rules to case-specific context and evidence. Written rules can leave gaps in decision guidance and process evaluation, while reference judgments vary in… 11 arXiv — NLP / Computation & Language research 5d ago SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction arXiv:2610.11559v1 Announce Type: new Abstract: Coding assistants such as Claude Code and Codex have become a major application of LLM agents, yet existing benchmarks remain far from real-world use, particularly in task horizon and interaction length. Code assistants require… 35 arXiv — NLP / Computation & Language research 5d ago Measuring Cultural Alignment Beyond the Average: A Framework for Evaluating Maternal-Health LLM Interactions in Indian Contexts arXiv:2610.11586v1 Announce Type: new Abstract: Existing evaluation methods for healthcare LLMs primarily assess factual correctness,safety, and fluency, while providing limited insight into whether generated interactions reflect culturally situated healthcare reasoning. This… 23 arXiv — NLP / Computation & Language research 5d ago Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing arXiv:2610.11599v1 Announce Type: new Abstract: Journals and conferences have begun to screen submitted manuscripts for text written using large language models (LLMs). The reliability of this screening rests on benchmark evaluations against a fixed set of LLM versions, while… 21 arXiv — NLP / Computation & Language research 5d ago TRACE: Diagnosing Verifier Brittleness in Agentic Evaluation arXiv:2610.11678v1 Announce Type: new Abstract: Verifier scores now serve as both benchmark metrics and training rewards for large language model (LLM) agents, and a change in score is routinely read as a change in capability. It may instead reflect a change in the evaluation.… 29 arXiv — NLP / Computation & Language research 5d ago Same Outcome, Different Evidence: Intent Recovery in LLM Safety Evaluation arXiv:2610.11766v1 Announce Type: new Abstract: Safety evaluations of large language models commonly summarize harmful-output behavior with attack success rate (ASR). Yet the same non-harmful outcome can arise for very different reasons. A model may recover a harmful task and… 26 arXiv — NLP / Computation & Language research 5d ago All Verdicts are Not Equal: Rethinking LLM Judge Reliability arXiv:2610.12083v1 Announce Type: new Abstract: LLM-as-a-Judge is the standard paradigm for NLP evaluation, yet its systemic reliability remains poorly understood despite being widely treated as a deterministic ground truth. We present a comprehensive reliability audit,… 34 arXiv — NLP / Computation & Language research 5d ago GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary arXiv:2610.11129v1 Announce Type: cross Abstract: Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge. Existing AI-Generated Game Commentary (AI-GGC) studies remain fragmented across games, modalities,… 29 r/MachineLearning community 5d ago ThinkingBox: Solving an agent task once vs. solving it 20/20: 507 stateful workflows graded on terminal database state [R] Figure 1b from our paper Disclosure: I'm one of the authors (Microsoft). The paper, code, dataset are public and ThinkingBox is on Hugging Face OpenEnv as well. Raw evaluation trajectories are not released Links at the bottom. We wanted to know how much of a single agent success… 22 TechCrunch — AI news-outlet 6d ago Popular AI leaderboard Arena nearly doubles valuation to $3.1B valuation in 10 months The company behind the popular LMArena leaderboard has raised $200 million led by Lightspeed and Khosla, and is now measuring AI models on alignment issues such as lying. 35 TechCrunch — AI news-outlet 6d ago Cal AI’s 19-year-old founder just raised $10M for his new AI startup Zach Yadegari, the teen co-founder of popular Cal AI calorie tracking app, has launched a new personal AI agent startup that competes with Instinct, Muse and Bee. 36 arXiv — Machine Learning research 6d ago Work While They Sleep: Exploiting Evaluation Latency for Fully Bayesian Optimization arXiv:2610.08969v1 Announce Type: new Abstract: Black-box optimization problems are ubiquitous across science and engineering, often dealing with expensive objective functions. This objective latency has two consequences during optimization: (i) the objective evaluation… 31 arXiv — Machine Learning research 6d ago SNR-Gated LSTM-Conditioned Diffusion Model for MIMO Channel Estimation arXiv:2610.08977v1 Announce Type: new Abstract: Accurate and low latency channel estimation is critical for modern MIMO systems, particularly under mobility, where channels exhibit structured sparsity and strong temporal correlation. This paper proposes a time-series conditioned… 36 arXiv — Machine Learning research 6d ago Conditional Accuracy Profiles: Diagnosing LLM Judges across Deployment Conditions arXiv:2610.09229v1 Announce Type: new Abstract: LLM-as-judge is now a standard tool for scalable evaluation, but judge performance is still often summarized by a single accuracy number. This aggregate view hides the deployment conditions under which a judge succeeds or fails. We… 22 arXiv — Machine Learning research 6d ago Efficient Best-of-N policy evaluation for inference-time alignment arXiv:2610.09250v1 Announce Type: new Abstract: Best-of-N (BoN) is a common inference-time alignment method that selects the highest-scoring response among N samples from a reference model. Evaluating BoN policies from logged data is challenging under sample-only access because… 15 arXiv — Machine Learning research 6d ago MORA: Modeling Observed Changes for Drift-Robust Time-Series Anomaly Detection arXiv:2610.09473v1 Announce Type: new Abstract: Time-series anomaly detection (TSAD) identifies deviations from patterns learned from historical data. In non-stationary settings, distribution drift and true anomalies can cause similar local changes, making it difficult to tell… 34 arXiv — Machine Learning research 6d ago Safe on Average, Unsafe in the Tail: When Is the Episodic-Cost Tail Controllable? arXiv:2610.09508v1 Announce Type: new Abstract: Safe reinforcement learning seeks policies that maximize return while satisfying constraints on cumulative cost. Most methods impose these constraints on expected episodic cost. Consequently, standard evaluations report mean… 15 arXiv — NLP / Computation & Language research 6d ago Tokka-Bench: Evaluating Tokenizers Across 100 Natural and 20 Programming Languages arXiv:2610.08794v1 Announce Type: new Abstract: Large language models rely on subword tokenizers whose quality varies across languages, yet no standardized multi-metric framework exists for broad comparative evaluation. We introduce Tokka-Bench, an open-source framework that… 19 arXiv — NLP / Computation & Language research 6d ago Quad-State Safety Evaluation of Open-Weight Large Language Models on Non-Canonical Inputs arXiv:2610.09033v1 Announce Type: new Abstract: Standard safety evaluations of large language models assess harmful requests written in canonical plain text, while models in real-world deployment routinely receive inputs containing emojis, altered spellings, encoded strings, and… 34 Page 1 of 10 · 500 articles Older →
