News / #multimodal Tag Multimodal 500 articles archived under #multimodal · RSS Sign in to follow llama.cpp releases dev-tools 11d ago b11361 llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) ( #29818 ) init conversion convert: ok model loaded add server code improve conversion script support shared prompt prefix add docs, imorove UX a bit add vision support add openjev tiny model for… 21 r/LocalLLaMA community 12d ago microsoft/FrogNano-4B-2609 · Hugging Face An agentic model from Microsoft for the GPU poor https://huggingface.co/bartowski/FrogNano-4B-2609-GGUF FrogNano is derived from Qwen/Qwen3.5-4B, a general-purpose post-trained model designed for language, reasoning, coding, agentic, and multimodal tasks. FrogNano inherits… 22 arXiv — Machine Learning research 12d ago Activation-Conditioned Self-Distillation arXiv:2609.38342v1 Announce Type: new Abstract: On-policy self-distillation uses a model as its own teacher to provide dense supervision for reasoning, often through reference-solution conditioning. Providing privileged information does not by itself ensure effective token-level… 6 arXiv — Machine Learning research 12d ago On the Off-Policy Teacher in On-Policy Distillation arXiv:2609.38360v1 Announce Type: new Abstract: On-policy distillation (OPD) has recently emerged as a promising post-training paradigm in which the student learns from trajectories generated by its own policy under dense teacher supervision. However, OPD introduces a… 22 arXiv — Machine Learning research 12d ago LeanPolish: Verified Supervision for Lean Proof Compression arXiv:2609.38384v1 Announce Type: new Abstract: Verified proof edits offer a natural source of supervision for improving language-model-generated Lean proofs. Yet verification establishes that an edit is correct, not that its training signal is free of search artifacts. We… 11 arXiv — Machine Learning research 12d ago Does Gradient Conflict Predict the Understanding--Generation Trade-off? A Controlled Audit of Conflict-Metric Validity in Unified Multimodal Models arXiv:2609.38465v1 Announce Type: new Abstract: Unified multimodal models (UMMs) are increasingly designed around gradient conflict between understanding and generation objectives. The premise that reducing these metrics improves the downstream understanding-generation trade-off… 11 arXiv — Machine Learning research 12d ago MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting arXiv:2609.38523v1 Announce Type: new Abstract: Financial forecasting from earnings conference calls requires models to reason over complex corporate disclosures, market expectations, and subtle communication signals. However, existing financial benchmarks are often limited to… 11 arXiv — Machine Learning research 12d ago ChartDensity-Bench: Benchmarking MLLMs for Numerical Data Reconstruction under Visual Density arXiv:2609.38781v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) offer a promising approach for recovering numerical data from scientific charts, but their ability to reconstruct chart data from visually dense figures remains poorly understood. Existing… 23 arXiv — Machine Learning research 12d ago LEARN-TS: LLM-Enhanced Alignment and Reconstruction with Normality Guidance for Multivariate Time-Series Anomaly Detection arXiv:2609.38789v1 Announce Type: new Abstract: Reconstruction errors in multivariate time-series anomaly detection may not reliably distinguish abnormal behavior from benign deviations. Language-derived semantics offer complementary context, but existing multimodal approaches… 27 arXiv — Machine Learning research 12d ago K2P: Label-Free Knowledge to Prompt Distillation arXiv:2609.38898v1 Announce Type: new Abstract: Knowledge distillation can transfer reasoning from stronger teachers to frozen students through reusable prompts, but avoiding weight updates does not eliminate supervision. Without ground-truth answers, teacher solutions are… 22 arXiv — NLP / Computation & Language research 13d ago Recovering Off-Policy Supervision for Speculative Decoding arXiv:2609.38795v1 Announce Type: new Abstract: Block drafters for speculative decoding are commonly trained on corpora written by external models, where a single off-policy token invalidates supervision for all subsequent slots in a block. Existing approaches discard these… 18 arXiv — NLP / Computation & Language research 13d ago Beyond Text: LLM-Based Dimensional Emotion Evaluation in Multimodal Dialogue arXiv:2609.39072v1 Announce Type: new Abstract: Emotion recognition in conversation has been widely studied, but applying Large Language Models (LLMs) to continuous dimensional emotion evaluation in multimodal dialogue remains largely unexplored. We propose an LLM-based… 25 arXiv — NLP / Computation & Language research 13d ago Spike-driven Vision-Language-Action Model arXiv:2609.39514v1 Announce Type: new Abstract: Vision-language-action (VLA) models bridge multimodal understanding and robotic control, advancing the dominant paradigm for embodied intelligence. However, most existing models rely on large Transformers, whose latency and energy… 23 arXiv — NLP / Computation & Language research 13d ago Better Supervision Is Nearby: Neighborhood On-Policy Self-Distillation arXiv:2609.39687v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard OPSD uses one fixed parameter setting at every… 6 arXiv — NLP / Computation & Language research 13d ago AdaGEPA: Adaptive Feedback Allocation for Reflective Prompt Optimization arXiv:2609.39927v1 Announce Type: new Abstract: Prompt optimization improves the performance of language-model systems on downstream tasks by refining their prompts. Classical methods evaluate prompts on task examples and use the resulting feedback to guide prompt revisions… 34 r/MachineLearning community 13d ago How to address novelty concerns in top ai conference? [D] Hi, I’m a researcher working in computer vision. Over the past few years, I’ve submitted several papers to top-tier conferences such as NeurIPS, ICLR, and CVPR, and one concern that seems to come up repeatedly is 'novelty'. Given that thousands of papers are published every year… 13 Stratechery (Ben Thompson) community 14d ago OpenAI Dev Day, Dot and OpenAI’s Product Transition, Sign In With ChatGPT OpenAI's Dev Day showcased a product that is, frankly, pretty confusing. However, there is more vision here than it might seem. 13 arXiv — Machine Learning research 14d ago Calibration-First Cross-Cohort Multimodal Temporal Learning for Transferable Asthma-Risk Forecasting arXiv:2609.35795v1 Announce Type: new Abstract: Asthma deterioration forecasting must remain reli- able when patient populations, sensor ecosystems, and available modalities change across cohorts. Existing models commonly optimize within-cohort discrimination and may produce… 13 arXiv — Machine Learning research 14d ago HeadGuard: Selective Head Protection for Low-Bit VLM KV-Cache Quantization arXiv:2609.35800v1 Announce Type: new Abstract: Low-bit key-value (KV) cache quantization saves storage but can sharply degrade vision-language model (VLM) accuracy. We introduce HeadGuard, a composable head-protection method that augments a base KV-cache quantizer with a fixed… 29 arXiv — Machine Learning research 14d ago ROSS: Relearning from Self-Generated Rollouts through Selective Supervision arXiv:2609.35954v1 Announce Type: new Abstract: Large language model post-training generates self-generated rollouts through reinforcement learning and on-policy distillation, yet this experience is often treated as stale once the policy advances. Historical rollouts can remain… 34 arXiv — Machine Learning research 14d ago Physical Cross-Modal Masked Autoencoding for Seismic-to-Well Representation Learning arXiv:2609.36193v1 Announce Type: new Abstract: Learning from scientific measurements often requires aligning modalities with different spatial support and resolution. Subsurface characterization is an extreme dense-sparse problem in which 3D seismic provides volumetric but… 9 arXiv — Machine Learning research 14d ago Paired Multimodal Scaling Laws arXiv:2609.36263v1 Announce Type: new Abstract: Existing multimodal scaling laws fit multimodality terms empirically after testing and never vary how much data is multimodally paired at fixed data budgets. We investigate how, under the same total data per modality, changing the… 35 arXiv — Machine Learning research 14d ago The Universal Classifier for Graph Learning arXiv:2609.36302v1 Announce Type: new Abstract: While foundation models have revolutionized natural language processing and computer vision by leveraging universal vocabularies, Graph Machine Learning (GML) remains fractured due to the absence of a unified feature and structural… 37 arXiv — Machine Learning research 14d ago AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models arXiv:2609.36368v1 Announce Type: new Abstract: Large foundation models have been introduced with the promise of efficient adaptation to downstream tasks. Yet, under limited supervision, MLLMs, an important class of large foundation models, remain challenging to adapt to various… 24 arXiv — Machine Learning research 14d ago Fisher-IRG: Fisher-Induced Local Invariant Representation Geometry across Language and Vision Models arXiv:2609.36458v1 Announce Type: new Abstract: Semantic-preserving transformations can induce substantial motion in learned representations, while small changes may strongly affect model predictions, raising a basic question: what local metric best captures semantically… 29 arXiv — NLP / Computation & Language research 14d ago Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News? arXiv:2609.35809v1 Announce Type: new Abstract: The rapid advancement of generative AI raises concerns about the misuse of Multimodal LLMs (MLLMs) for large-scale disinformation campaigns on social media. Despite existing research on textual disinformation, a fundamental… 26 arXiv — NLP / Computation & Language research 14d ago PrimeSeeker: Capability-Oriented Supervision for Deep Search Agents arXiv:2609.35816v1 Announce Type: new Abstract: Large language model search agents are often trained with synthetic questions whose difficulty is increased through larger evidence graphs, additional hops, and longer trajectories. These global properties, however, are only… 20 arXiv — NLP / Computation & Language research 14d ago When Should LLMs Trust Their Own Revisions? A Risk-Aware Study of Intrinsic Self-Correction arXiv:2609.35832v1 Announce Type: new Abstract: Intrinsic self-correction asks a language model to revise its own answer without receiving new external evidence. A second pass can recover mistakes, but it can also overturn answers that were already correct. We study this… 5 arXiv — NLP / Computation & Language research 14d ago Question-Specific Knowledge Graphs for Efficient Visual Reasoning arXiv:2609.35942v1 Announce Type: new Abstract: Recent work in visual question answering has shown that vision-language models can exhibit strong reasoning capabilities by translating visual inputs into textual representations. The effectiveness of this translation depends on… 17 arXiv — NLP / Computation & Language research 14d ago DeepRewind: Predicting and Repairing Premature Commitments in Deep Research Agents arXiv:2609.36344v1 Announce Type: new Abstract: Deep-research agents conduct long-horizon investigations through iterative search, evidence evaluation, belief revision, and synthesis. However, they may commit to claims before sufficient evidence is available, causing later… 38 arXiv — NLP / Computation & Language research 14d ago Similar Choices, Different Attention: Cross-Modal Associations in Humans and Vision-Language Models arXiv:2609.36475v1 Announce Type: new Abstract: Cross-modal associations are systematic pairings of features across modalities, such as the association of 'bouba' with round shapes and 'kiki' with sharp shapes. Prior work has compared humans and vision-language models (VLMs) on… 18 arXiv — NLP / Computation & Language research 14d ago Grounded Revision vs. Prior Injection: Probing Retrieval-Augmented Patent Claim Amendment arXiv:2609.36550v1 Announce Type: new Abstract: Retrieval-augmented generation is widely used in professional writing, yet whether retrieval grounds revision or merely injects templates is rarely tested where "correct" has a definable meaning. Patent claim amendment supplies… 7 arXiv — NLP / Computation & Language research 14d ago Rethinking Multimodal Fake News Detection in the Generative AI Era arXiv:2609.36850v1 Announce Type: new Abstract: Generative content is increasingly entering the production and dissemination of news, transforming fake news from manually fabricated or simply manipulated material into complex forms in which native and generated content jointly… 7 arXiv — NLP / Computation & Language research 14d ago RAEGNet: Relation-Aware Evidence Graph Network for Harm-Aware Multimodal Fake News Detection arXiv:2609.36902v1 Announce Type: new Abstract: Existing multimodal fake news detection methods often introduce external information to assist detection. However, most of them rely on entity-level retrieval and are therefore prone to introducing event-irrelevant noise.… 22 The Information — AI news-outlet 15d ago President Trump Announces ‘Super Intelligence’ Accord, Renames AI President Donald Trump announced on Tuesday that leading AI companies had agreed to non-binding commitments on AI safety provisions, including internal controls on model alignment and external auditing. The “White House Accord on Super Intelligence,” which Trump posted on Truth… 21 NVIDIA Developer Blog official-blog 15d ago Lower the Cost of Building and Running Visual AI Agents with NVIDIA VSS Blueprint 3.3 Vision-language models have made it possible to build visual AI agents that understand video at production scale. The harder problem is turning that capability... 5 r/LocalLLaMA community 15d ago RAM Offloading with vLLM - tcclaviger appreciation post Thanks to tcclaviger, vLLM now has expert RAM offloading support ( link ). This makes frontier models much more accessible on a local setup! I was able to run the original DeepSeek-V4-Flash-Vision-Exp on four R9700s. podman run --rm -it \ --init \ --network host \ --ulimit… 27 arXiv — Machine Learning research 15d ago Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysis arXiv:2609.30470v1 Announce Type: new Abstract: Multimodal Sentiment Analysis (MSA) aims to infer human emotions from multiple modalities such as text, audio, and vision. In practice, inputs are often corrupted by noise and missing modalities, which degrades performance.… 20 arXiv — Machine Learning research 15d ago QSV: Quat-Sphere-Vision for Coupled Quaternion Attention on Spherical Lattices arXiv:2609.30592v1 Announce Type: new Abstract: In standard attention, three separately learned projections decide how strongly a token attends to each neighbor ($W_Q$, $W_K$) and how the attended features are transformed before aggregation ($W_V$). We study Quat-Sphere-Vision… 28 arXiv — Machine Learning research 15d ago The Linear Representation Hypothesis for Vision-Language-Action Models arXiv:2609.30996v1 Announce Type: new Abstract: The linear representation hypothesis (LRH) has become a standard lens for measuring and intervening on semantic information through the internal representations of large language models (LLMs). A growing body of work has begun… 19 arXiv — Machine Learning research 15d ago From Shortcut Learning to Discrete Neural Insertion Sort arXiv:2609.31114v1 Announce Type: new Abstract: Neural algorithmic reasoning aims to train neural networks to follow known algorithms and generalize beyond the input sizes seen during training. However, correct final outputs and intermediate supervision do not necessarily show… 24 arXiv — Machine Learning research 15d ago WorldTS: World Modeling for Multimodal Covariate-aware Time Series Forecasting arXiv:2609.31162v1 Announce Type: new Abstract: Time series forecasting is typically framed as learning a direct mapping from historical to future observations in the observation space. However, sequences of observations generally provide only a partial view of the dynamics of… 17 arXiv — Machine Learning research 15d ago Self-Supervised Representation Learning: From Spectral Foundation Models to Auroral Emission Spectra arXiv:2609.31206v1 Announce Type: new Abstract: Auroral spectrographs such as the Auroral Spectrograph In Skibotn (ASIS) record hundreds of thousands of emission spectra, but only a few hundred can be labelled by an expert. To exploit the rest, we pretrain a 1D Vision… 21 llama.cpp releases dev-tools 16d ago b11240 server : support typed content (vision/audio/video) input for /v1/embeddings endpoint ( #29556 ) server : support multimodal input for /v1/embeddings (Qwen3-VL-Embedding) Accept the OpenAI-style wrapped content array format for multimodal embedding requests. Each {"content":… 10 The Information — AI news-outlet 16d ago Meta Taps MongoDB CEO to Lead New Enterprise AI Division Meta Platforms is launching a new division to sell its AI tools to businesses, tapping MongoDB Chief Executive Chirantan Desai to lead the unit as chief enterprise platform officer. In a post on X Monday, Meta CEO Mark Zuckerberg said the new division, called Meta Enterprise… 12 r/LocalLLaMA community 16d ago Holo4 Holo4 -27B-GGUF is a vision-language model (VLM) for Computer Use, built on the Qwen3.8 dense architecture and developed by H Company . Used with the hai-agents harness , it can send screenshots and tool results to the model, then execute its requested clicks, typing, code, and… 22 llama.cpp releases dev-tools 16d ago b11227 context : do not re-reserve the scheduler when toggling causal_attn ( #28751 ) context : do not re-reserve the scheduler when toggling causal_attn llama_context::set_causal_attn() marks the scheduler to do a full re-reserve on every change of the flag. For vision inputs, this… 27 arXiv — NLP / Computation & Language research 16d ago Recursive Self-Improvement via On-Policy Distillation for Reasoning arXiv:2609.30652v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student model by having it generate trajectories, then matching its next-token predictions with an external teacher's next-token predictions. This provides dense, token-level supervision to the… 15 arXiv — NLP / Computation & Language research 16d ago SEA-CLIP-Tiny: Efficient Multilingual Text-Vision Embedding for Southeast Asian Languages arXiv:2609.30739v1 Announce Type: new Abstract: Multilingual text-vision embedding models are essential for cross-lingual image-text retrieval, but Southeast Asian languages remain poorly supported due to the region's linguistic diversity and limited data and computing… 10 arXiv — NLP / Computation & Language research 16d ago Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weighting arXiv:2609.31169v1 Announce Type: new Abstract: Multimodal Machine Translation aims to incorporate additional signal from non-textual modalities to improve translations by resolving ambiguities. While models, through multimodal fusion, are able to accept images related to the… 16 Page 3 of 10 · 500 articles ← Newer Older →
