arXiv — Machine Learning
500 articles archived · Visit source ↗ · RSS
-
arXiv — Machine Learning research 2d ago
Algebraic Integer Quantization for Machine Learning Applications
arXiv:2610.12494v1 Announce Type: new Abstract: In this article, we propose algebraic integer number formats (AINF) as a framework for highly configurable and accurate compressed computation. A key property of AINF is its straightforward reduction to purely integer arithmetic,…
12 -
Kairos: Toward a Byte-Level Foundation Model for Every Modality
arXiv:2610.12495v1 Announce Type: new Abstract: Small language models trained on a single local machine leave little room for architectural trial and error. Kairos combines bidirectional gated-DeltaNet linear attention and sliding-window attention over shared projections…
27 -
An Input-Dependent Readout Phase for Resonant Sequence Models
arXiv:2610.12496v1 Announce Type: new Abstract: Sequence models with a fixed-size state avoid the quadratic cost of attention, but face a design choice. An input-dependent state transition gives content-dependent memory at the cost of FFT-based training, while a fixed transition…
25 -
CircuitTree: Composing Circuits into A Syllable Transcriber
arXiv:2610.12520v1 Announce Type: new Abstract: The goal of circuit discovery is to decompose a monolithic neural network into tiny modular subnetworks which are interpretable in the sense that each does an identifiable subtask. We develop a practical method for composing…
6 -
Behind-the-Meter Photovoltaic Visibility from Net Smart Meters: Deployment Regimes and Operational Insights
arXiv:2610.12527v1 Announce Type: new Abstract: Distribution operators need visibility of behind-the-meter photovoltaic (PV) generation, yet most residential customers are observed only through net smart meters. Deep-learning-based PV disaggregation can restore this visibility,…
29 -
GlucoPRISM: Protocol-Supervised Pretraining for Cross-Cohort Glucose Transfer
arXiv:2610.12528v1 Announce Type: new Abstract: Continuous glucose monitoring (CGM) foundation models encode each day as one vector that mixes who the person is, what happened that day and how the sensor measured it, so measurement differences travel with the representation into…
20 -
CacheScope: Exact-Compute Allocation for Training-Free Diffusion Acceleration
arXiv:2610.12530v1 Announce Type: new Abstract: Training-free feature reuse accelerates diffusion transformer inference by replacing full denoising computation at selected steps with cached approximations. Existing methods mainly improve the reconstruction of skipped features,…
18 -
Market Impact Estimation via Causal Foundation Models
arXiv:2610.12537v1 Announce Type: new Abstract: When a trader executes a large order, the relevant question is not \emph{what will the price be?} but \emph{what would the price have been had I not traded?} Market impact is thus a causal quantity, yet classical models…
16 -
Efficient Learning of Survival Analysis Models Under Outcome-Dependent Truncation and Censoring
arXiv:2610.12538v1 Announce Type: new Abstract: Modern machine learning models are typically trained on data assumed to be representative of the predictive tasks the model will face in the future, but practical datasets violate this assumption. We focus on outcome-dependent…
4 -
Enriched Holomorphic Neural Networks for 2D and 3D Laplace Problems
arXiv:2610.12541v1 Announce Type: new Abstract: Boundary-value problems for the Laplace equation develop singular behaviour near corners, edges and junctions that smooth approximators resolve poorly. We combine holomorphic neural networks with singular functions derived from the…
17 -
Transformer-Based Masked Diffusion Models Provably Learn Distributions with Latent Structure
arXiv:2610.12544v1 Announce Type: new Abstract: Masked diffusion models (MDMs) generate discrete data by iteratively unmasking multiple tokens in parallel using a learned mask predictor. Their output carries two errors: parallelization error due to parallel sampling, and…
7 -
Sample-Adaptive Zeroth-Order Post-Training
arXiv:2610.12566v1 Announce Type: new Abstract: Zeroth-order (ZO) optimization offers a memory-efficient alternative to backpropagation for fine-tuning large language models. However, existing ZO methods typically sample training examples uniformly, overlooking their evolving…
22 -
An all-transistor physical-learning transformer can be trained to perform in-context learning
arXiv:2610.12571v1 Announce Type: new Abstract: Most systems trained by equilibrium propagation or coupled learning are in the same class as multi-layer perceptrons: while they may be universal function approximators, they struggle to learn associative tasks which are easy for…
37 -
DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models
arXiv:2610.12572v1 Announce Type: new Abstract: Online reinforcement learning (RL) fine-tunes diffusion models with black-box rewards, but reverse-process methods such as Flow-GRPO are limited in efficiency by poor credit assignment, while forward-process methods such as…
34 -
Realism VS Accuracy: Event Sequence Forecasting from a Generative Modeling Perspective
arXiv:2610.12586v1 Announce Type: new Abstract: Event sequences are chains of random events, each described by heterogeneous (numerical and categorical) features. The problem of event sequence forecasting occurs in finance, healthcare, business process intelligence, and other…
36 -
STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules
arXiv:2610.12599v1 Announce Type: new Abstract: Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures remains expensive and data-hungry. Existing sparse upcycling methods mainly…
17 -
Machine Learning and Time-Series Foundation Models for Day-Ahead Electricity Forecasting in a University Building
arXiv:2610.12638v1 Announce Type: new Abstract: Day-ahead electricity forecasts can help building operators schedule heating, ventilation, and other services around anticipated demand. Choosing a forecasting model also requires balancing accuracy with its forecast-time inputs,…
24 -
SynAct: Synergistic Active Feedback Learning for LLM Agents
arXiv:2610.12651v1 Announce Type: new Abstract: LLM agents executing multi-step workflows fail in two fundamentally different ways: recoverable slips, which further attempts can repair, and knowledge gaps, which retrying cannot resolve. Expert feedback can address the latter,…
29 -
Kronecker Product Attention: Memory-Efficient KV Compression via Structured Factorization
arXiv:2610.12657v1 Announce Type: new Abstract: Efficient autoregressive generation is increasingly limited by the memory footprint of the key-value (KV) cache, which grows linearly with sequence length, batch size, and model depth. Existing approaches such as Multi-Query…
28 -
Data-driven Optimal Sensor Placement for Aerosol Monitoring in the UK
arXiv:2610.12658v1 Announce Type: new Abstract: We develop a data-driven sensor-placement framework and apply it to aerosol monitoring in the United Kingdom (UK). A trained convolutional Gaussian neural process is embedded within the DeepSensor optimisation framework. Genetic…
24 -
Recurrent Harness for Agentic Continual Learning
arXiv:2610.12668v1 Announce Type: new Abstract: Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing agent harnesses typically either evolve their states only…
6 -
Amortised Inference through One-Step Implicit Sampling
arXiv:2610.12679v1 Announce Type: new Abstract: Efficient sampling from Bayesian posterior distributions in high dimensions remains a hard problem, despite advances in theory and algorithms. Multiple steps are usually required to draw unbiased samples, for example, iterations of…
24 -
Expert-level detection of low-voltage intervals in EEG as a robust biomarker of neonatal encephalopathy
arXiv:2610.12680v1 Announce Type: new Abstract: Background: Discontinuity in neonatal EEG is a critical marker of brain dysfunction but can be difficult to quantify visually. Inter-burst intervals (IBIs) are a key characteristic of discontinuity, yet their definition does not…
35 -
Critical Mass: Structure-Disjoint Splits Drift With the Libraries They Are Built On
arXiv:2610.12694v1 Announce Type: new Abstract: Molecular machine learning evaluates out-of-distribution performance with structure disjoint splits, a construction rule fixed once at the scale its library happened to have: threshold a pairwise distance, then assign whole…
25 -
Primal-Dual Flow Matching for Sample-Wise Constrained Generation
arXiv:2610.12699v1 Announce Type: new Abstract: We consider the problem of learning to generate data while satisfying a prescribed \emph{sample-wise} constraint with a user-specified probability of constraint satisfaction. Starting with a population-level constrained negative…
25 -
Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passing
arXiv:2610.12711v1 Announce Type: new Abstract: Federated Large Language Models (FedLLMs) are promising to deliver privacy-preserving, collaborative intelligence. The decentralized training process is vulnerable to Byzantine attacks that corrupt semantic consistency and factual…
24 -
Spatial Context or Memory? Sizing the Inputs of Neural Surrogates for Partially Observed Fields
arXiv:2610.12723v1 Announce Type: new Abstract: Neural surrogates that forecast a physical field from sparse, noisy sensors read a window of nearby sensors and a history of past snapshots. Practitioners choose both by comparing fitted models with each other. Such comparisons…
11 -
A Categorical Atlas of Parameter-Efficient Fine-Tuning: Cones, Orbits, and Gauges
arXiv:2610.12728v1 Announce Type: new Abstract: Parameter-efficient fine-tuning has hundreds of methods, usually compared by parameter count and benchmark score. We compare them as maps. A pretrained checkpoint is a base point in weight space, and a weight-space method is a…
21 -
Kernelized Parameter Sensitivity Atlases for Amortized Conditional Interpretability
arXiv:2610.12737v1 Announce Type: new Abstract: Gradient-based attribution provides input-specific information about model parameters, but computing it via backpropagation for many samples is expensive, while static importance scores discard input dependence. We derive KPSA…
37 -
KITE: A Knowledge-Injected Temporal Encoder for Time-Series Event Forecasting
arXiv:2610.12738v1 Announce Type: new Abstract: Forecasting rare, high-cost events in multivariate time-series systems is hard because their precursors are seldom observed, so a purely data-driven model has little to learn from. The engineers who run these systems, however, hold…
27 -
Neural expectation-maximization and meta-models
arXiv:2610.12741v1 Announce Type: new Abstract: We interpret Transformer networks as learned approximations to a hybrid probabilistic-inference procedure that combines max-product Loopy Belief Propagation (LBP) with Expectation--Maximization (EM). In this view, a Transformer…
14 -
-
NV-Tesseract-AD: Curriculum-Enhanced Conditional Diffusion for Multivariate Time-Series Anomaly Detection
arXiv:2610.12754v1 Announce Type: new Abstract: We propose NV-Tesseract-AD, a curriculum learning-enhanced conditional diffusion framework for multivariate time-series anomaly detection. Our central hypothesis is that reconstruction difficulty can be explicitly structured along…
38 -
Provenance-grouped protocol for evaluating machine learning in perovskite photovoltaics
arXiv:2610.12766v1 Announce Type: new Abstract: Machine-learning models for perovskite solar cells are typically evaluated using random data splits, which we show systematically leak information through massive internal duplication. Because nearly two-thirds of curated devices…
37 -
REDaCT: Collapse-Free Unlearning in Large Reasoning Models
arXiv:2610.12769v1 Announce Type: new Abstract: Unlearning in large reasoning models (LRMs) requires removing forget-set information from both the reasoning trace and the final answer. Existing approaches achieve this objective at the cost of reasoning collapse, where the model…
18 -
Coordinate-Free Automatic Differentiation
arXiv:2610.12776v1 Announce Type: new Abstract: Automatic differentiation (AD) is a cornerstone of modern machine learning. Since all AD frameworks implicitly fix a basis of Euclidean space to define gradients and Jacobians, they natively process mathematical objects that are…
15 -
Meta-Reflective Evolution Effectively Improves Production AI Systems
arXiv:2610.12794v1 Announce Type: new Abstract: Large Language Model (LLM)-guided evolutionary optimization has successfully improved prompts, programs, and even entire AI systems. Most optimization methods select a candidate from the search frontier, then mutate it in one LLM…
23 -
The Expressive Power of Random-Feature MPNNs Across Graph Sizes
arXiv:2610.12809v1 Announce Type: new Abstract: Continuous random node features increase the expressivity of message-passing neural networks (MPNNs), but existing theoretical analyses are not fine-grained enough to capture their limitations, particularly in approximating…
16 -
Deploying RDMA for Large-Scale Inference and Training Workloads
arXiv:2610.12815v1 Announce Type: new Abstract: RDMA can reduce communication overhead in distributed machine learning systems, but realizing these benefits in production requires more than replacing TCP. Variable-size messages, concurrent connections, bursty incasts, and…
24 -
-
TreeHCA: Hindsight-Guided Exploration and Credit Assignment for Multi-Turn LLM Agents
arXiv:2610.12828v1 Announce Type: new Abstract: Outcome-supervised group-relative policy optimization (GRPO) assigns the same advantage to every token in an LLM agent trajectory, leaving useful and unhelpful decisions undifferentiated. Hindsight credit assignment (HCA) can…
33 -
-
When Generative Models Act as Simulators
arXiv:2610.12847v1 Announce Type: new Abstract: A simulator summarizes an environment in a state that users can read and change. Generative models learn representations of the world without explicit specification, but offer no built-in way to edit that world's state. We…
5 -
Reinforcement Learning via Fixed-point Flow Matching
arXiv:2610.12848v1 Announce Type: new Abstract: In maximum entropy and behavior-constrained reinforcement learning (RL), the optimal policy is given by a Boltzmann distribution. Generating samples from this distribution can be challenging, as it is often complex and multimodal.…
32 -
Robust Langevin Sampling under State-Dependent Heavy-Tailed Noise with Applications to LLM Fine-Tuning
arXiv:2610.12865v1 Announce Type: new Abstract: Existing convergence guarantees for non-log-concave sampling with stochastic gradient Langevin Monte Carlo (SG-LMC) typically assume globally Lipschitz gradients and uniformly bounded noise variance. Both assumptions can fail even…
17 -
HART: Hybrid Attention-RWKV Time series Foundation Model
arXiv:2610.12883v1 Announce Type: new Abstract: Non-autoregressive multi-step forecasting (NAF) enables time series foundation models to predict an entire future horizon in a single forward pass. Despite its strong sequence modeling capabilities, RWKV-7 exhibits long-horizon…
17 -
CoGameReg: Cooperative-Game-Based Collaborative Semi-Supervised Regression for Seismic Reservoir Property Prediction
arXiv:2610.12884v1 Announce Type: new Abstract: Accurate estimation of reservoir properties from seismic data remains challenging when well-log labels are sparse but three-dimensional seismic observations are abundant. We present CoGameReg, a cooperative-game-based…
20 -
Personalized Federated Direct Preference Optimization via LoRA Adapters
arXiv:2610.12885v1 Announce Type: new Abstract: Federated Direct Preference Optimization (FedDPO) enables privacy-preserving alignment of language models across distributed agents without centralizing sensitive preference data. However, existing FedDPO analysis assumes all…
24 -
GIST: Generator-state-Informed Supervision for Tabular Foundation Models
arXiv:2610.12904v1 Announce Type: new Abstract: Tabular foundation models (TFMs) based on prior-data fitted networks typically use sampled hard query labels, leaving avoidable sampling variability in pretraining. A key observation is that intermediate generator states can reveal…
27 -
Controlled-Minimax: Stabilizing Adversarial Optimization via Control Theory
arXiv:2610.12915v1 Announce Type: new Abstract: Minimax optimization is fundamental to adversarial learning yet suffers from severe stability challenges, often manifesting as persistent limit cycles or divergence. This paper investigates these instabilities through the lens of…
6