News / #multimodal Tag Multimodal 500 articles archived under #multimodal · RSS Sign in to follow llama.cpp releases dev-tools 1mo ago b10766 model: correctly support input vision for deepseek4 ( #28154 ) model: correctly support input vision for deepseek4 nits Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/44819834 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon… 6 llama.cpp releases dev-tools 1mo ago b10762 mtmd: support DeepSeek-V4-Flash-Vision-Exp ( #28133 ) mtmd: support DeepSeek-V4-Flash-Vision-Exp handle min/max token counts from CLI rm debugging use GGML_ROPE_TYPE_VISION nits apply review comments correct token count Website: https://llama.app Attestations:… 29 r/LocalLLaMA community 1mo ago Vision support merged for DeepSeek-V4-Flash-Vision-Exp Unsloth GGUFs and Vision support https://huggingface.co/unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF   submitted by   /u/fmillar [link]   [comments] 36 Hugging Face Daily Papers research 1mo ago Learning Where Outcomes Change:Credit-Addressable Reasoning for Multimodal Geometry Abstract Credit-addressable reasoning via executable code traces and localized reinforcement learning improves multimodal geometry reasoning by aligning credit assignment with structured reasoning events. Generated by thinkingmachines/Inkling-Small Multimodal geometry reasoning… 5 Hugging Face Daily Papers research 1mo ago Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving Abstract Qwen-Drive-1.0 is a vision-language foundation model for autonomous driving that unifies 3D perception, visual question answering, and motion planning via shared representations and staged training. Generated by thinkingmachines/Inkling-Small We present Qwen-Drive-1.0,… 29 arXiv — Machine Learning research 1mo ago Good Memory Has ECC: Evaluating the Memory of Vision-Language Models Beyond Accuracy arXiv:2609.00103v1 Announce Type: new Abstract: Memory is widely viewed as an important unsolved problem for LLMs and VLMs, and current benchmarks typically evaluate it by testing accuracy over long text or video. However, accuracy alone misses properties that matter for real… 23 arXiv — Machine Learning research 1mo ago CRAFT: Fine-Tuning Pre-hoc Explainability in AI-native 6G RAN arXiv:2609.00590v1 Announce Type: new Abstract: The next generation of mobile networks is envisioned as fully AI-native, with AI-RAN architectures embedding small language models (SLMs) to perform reasoning over real-time telemetry. The state-of-the-art training paradigms for… 26 arXiv — Machine Learning research 1mo ago Text Capability Loss in Vision-Language Adaptation: An Attention-Sink Diagnosis arXiv:2609.00746v1 Announce Type: new Abstract: Fine-tuning a pretrained LLM into a vision-language model (VLM) can erode the backbone's text capability, with the damage concentrated on tasks that require following exact output rules, such as instruction following,… 17 arXiv — Machine Learning research 1mo ago REFACTOR-VLA: Unsupervised Library Learning of Typed Motor Programs arXiv:2609.01215v1 Announce Type: new Abstract: Most vision-language-action (VLA) models -- OpenVLA, $\pi_0$, RT-2, RDT-1B -- are monolithic: they emit raw motor commands or short action chunks without organizing behavior into reusable abstractions, so they degrade on… 38 arXiv — Machine Learning research 1mo ago Contribution-Aware Bandwidth Allocation for Multimodal Split Learning arXiv:2609.01406v1 Announce Type: new Abstract: Multimodal models are increasingly the default option for perception at the network edge, yet they are trained almost entirely in the datacenter, because a client holding several sensor streams cannot host an encoder per modality.… 24 arXiv — NLP / Computation & Language research 1mo ago Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancy arXiv:2609.00067v1 Announce Type: new Abstract: External text can override conflicting image evidence in multimodal large language models, a failure we call multimodal contextual sycophancy. We introduce a 998-case diagnostic that independently varies visual evidence,… 5 arXiv — NLP / Computation & Language research 1mo ago Uncovering and Mitigating Aggregation-Induced Reward Hacking in Multi-Reward Reinforcement Learning arXiv:2609.00213v1 Announce Type: new Abstract: Reinforcement learning fine-tuning of large language models increasingly adopts multiple reward dimensions, including verifiable rules, task-specific evaluators, and learned reward models, to provide richer supervision across… 30 arXiv — NLP / Computation & Language research 1mo ago Slow to See, Slow to Suppress: Understanding the Effects of Modality in Context-Memory Conflicts arXiv:2609.00293v1 Announce Type: new Abstract: We investigate how vision-language models (VLMs) handle context-memory conflicts; that is, situations in which the model is given information in context that differs from what was stored parametrically during training. We document… 30 arXiv — NLP / Computation & Language research 1mo ago (V)LMs generalize beyond surface co-occurrence: Evidence from cross-modal number agreement arXiv:2609.00443v1 Announce Type: new Abstract: Language models learn about grammatical number primarily from co-occurrence, and show frequency effects as a result---sometimes taken to indicate that they do not learn abstract ``rules'', and are instead dependent on specific… 9 arXiv — NLP / Computation & Language research 1mo ago Same Semantics, Different Outcome: On the Modality Robustness of Multimodal LLMs under Knowledge Conflict arXiv:2609.00550v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly provided with contextual evidence in heterogeneous forms: as a text passage, as a rendered image of the same passage, or as both together. However, it remains unclear how… 27 arXiv — NLP / Computation & Language research 1mo ago EM^2Mem: Event-Centric Multimodal Memory for Large Language Models arXiv:2609.00551v1 Announce Type: new Abstract: Multimodal memory offers a scalable interface for long-video question answering, but existing methods often retrieve captions, frames, transcripts, summaries, or graph facts as isolated fragments. Although searchable, such… 16 arXiv — NLP / Computation & Language research 1mo ago ExpArt-KG: Artwork Image Description Generation through Iterative Exploration of Knowledge Graphs arXiv:2609.00629v1 Announce Type: new Abstract: Large Vision-Language Models (LVLMs) achieve strong performance on image-grounded text generation and visual question answering. However, it remains difficult for them to comprehensively and accurately describe the factual… 28 arXiv — NLP / Computation & Language research 1mo ago Joint Training Is Not Enough: Conditioned Cross-Granularity Training for Multimodal Document Understanding arXiv:2609.00756v1 Announce Type: new Abstract: The Mutual Reinforcement Effect (MRE) asks whether a fine, span-level and a coarse, document-level task help each other when one model handles both. We test it in multimodal document understanding on three corpora, two of receipts… 9 arXiv — NLP / Computation & Language research 1mo ago Dense Process Supervision for Search Agents via Fact Utility Estimation arXiv:2609.00833v1 Announce Type: new Abstract: Reinforcement learning (RL) for search agents typically relies on outcome rewards. However, it often fails to achieve effective credit assignment, due to the unclear value of intermediate steps. It is hard to separate their… 32 arXiv — NLP / Computation & Language research 1mo ago PersianAnonymizer: Evaluating LLM-Labeled Training for Efficient NER-based Anonymization in Persian arXiv:2609.00958v1 Announce Type: new Abstract: We target practical anonymization of Persian customer chats by training a compact NER model from LLM-labeled supervision and selecting the best labeler for deployment. We compare three instruction-tuned LLMs: DeepSeek-V3-0324,… 27 arXiv — NLP / Computation & Language research 1mo ago OUTLETS: Output-Length Prediction from Speculative Decoding Backbones arXiv:2609.01068v1 Announce Type: new Abstract: The heavy-tailed distribution of output lengths in Large Language Model (LLM) serving poses major challenges for resource provisioning and cluster scheduling. Although output-length prediction can mitigate these issues, existing… 17 arXiv — NLP / Computation & Language research 1mo ago When Modality Gap Reduction Fails: Prediction-Level Hubness in CLIP arXiv:2609.01103v1 Announce Type: new Abstract: Reducing the modality gap between image and text representations in CLIP is widely expected to improve cross-modal alignment and downstream performance. However, a smaller average image-text gap does not necessarily lead to… 27 Hugging Face Daily Papers research 1mo ago UI-Venus-2 Technical Report Abstract UI-Venus-2 is a general-purpose multimodal GUI agent that uses unified reasoning-action loops, expanded environment coverage, and robust verification to enable reliable real-world digital automation. Generated by thinkingmachines/Inkling-Small Multimodal GUI agents have… 34 Hugging Face Daily Papers research 1mo ago Evaluating Multimodal LLMs as Generalist Vision-Language-Action Agents for Drone Control: Commanding, Approaching, Tracking and Searching Abstract DroneCATS evaluates multimodal language models as drone controllers and finds that small open models navigate well but fail at protocol adherence and termination, highlighting a gap between spatial perception and disciplined action. Generated by… 7 Hugging Face Daily Papers research 1mo ago Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System Abstract Unified multimodal models achieve synergy between visual understanding and generation through specialized architectures, shared knowledge, and end-to-end optimization rather than simple functional unification. Generated by thinkingmachines/Inkling-Small While unified… 29 Hugging Face Daily Papers research 1mo ago EM^2Mem: Event-Centric Multimodal Memory for Large Language Models Abstract EM²Mem binds multimodal evidence to event anchors for compact, generation-ready memory in long-video question answering. Generated by thinkingmachines/Inkling-Small Multimodal memory offers a scalable interface for long-video question answering, but existing methods… 4 Hugging Face Daily Papers research 1mo ago ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation Abstract ReFlowSET selects a latent codec via joint SAR-EO reconstruction and trains a small conditional DiT with dual-stream conditioning and frozen vision-model alignment for high-fidelity SAR-to-EO translation. Generated by thinkingmachines/Inkling-Small SAR-to-EO image… 15 r/LocalLLaMA community 1mo ago Gave a try to Exllamav3 and it's great! Following this post I decided to try GLM 5.3 Flash on a 8x3090 setup and I can now run a Q4 with surprising speed; 700tk/s prefill & 42tk/s decoding! (lcp & vllm do not allow me to get that). Was afraid about quality but > 30m tokens with DSH and no issue (did not test vision… 36 Hugging Face Daily Papers research 1mo ago CoVA-SFT: A Large-Scale Dataset for Chain of Visual Abstractions Abstract CoVA-SFT is a multimodal reasoning dataset that teaches language models to interleave text and visual abstractions via structured reasoning steps, improving performance on visual reasoning benchmarks. Generated by thinkingmachines/Inkling-Small Chain-of-thought (CoT)… 9 Hugging Face Daily Papers research 1mo ago MMMMM: A Unified Taxonomy for Investigating the Mechanisms of Multilingual MultiModal Misinformation Abstract Multimodal misinformation on social media is highly prevalent, potent, and harmful, yet difficult to detect and counter, and still poorly understood compared to its text-only counterpart. Research on the properties and deceptive strategies of multimodal misinformation… 34 Hugging Face Daily Papers research 1mo ago SpanCalib-VLM: Calibrated Hallucination Span Detection in Vision-Language Models Abstract A hybrid system combining a multimodal sequence tagger and a generative vision-language model improves hallucination span detection and calibration through union-calibrated fusion. Generated by thinkingmachines/Inkling-Small Detecting hallucinations in Large… 15 r/LocalLLaMA community 1mo ago All currently popular local models in one table + Opus 4.8 results If you are thinking what model will fit best your HW specs and tasks you are doing here is one table with all currently popular models that still can be considered as local. LLM Test Scores Feature DeepSeek-V4-Flash-Vision-Exp DeepSeek-V4-Flash-0731 Qwen3.8-Flash-Next… 24 Hugging Face Daily Papers research 1mo ago SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models Abstract SafeAtlas-VL introduces a large-scale multimodal safety dataset with graded risk labels and trains guard models that predict continuous safety scores, achieving state-of-the-art generalization. Generated by thinkingmachines/Inkling-Small Multimodal safety moderation… 30 Hugging Face Daily Papers research 1mo ago Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation Abstract NavMCP integrates vision-language reasoning with navigation execution via structured collaboration channels to enable persistent long-horizon embodied exploration without retraining. Generated by thinkingmachines/Inkling-Small Long-horizon physical-world agents must… 35 Hugging Face Daily Papers research 1mo ago PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback Abstract A multi-turn benchmark and multi-agent system improve scientific diagram generation by reducing quality drift and feature forgetting across revision turns. Generated by thinkingmachines/Inkling-Small Recent efforts have aimed to automate scientific diagram generation… 22 Hugging Face Daily Papers research 1mo ago LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation Abstract LightNav-0 is a compact generalist navigation model that leverages a pretrained vision-language model’s spatial reasoning via unified pointing tokens and action tokenization to achieve state-of-the-art embodied navigation across diverse tasks and robots. Generated by… 34 Hugging Face Daily Papers research 1mo ago WebWorld: The Browser as a World Model for Self-Improving Web Code Abstract WebWorld uses a browser as an executable world model to certify web code repairs via interaction contracts, enabling autonomous VLM self-improvement with verified supervision. Generated by thinkingmachines/Inkling-Small VLM-driven self-improvement of web code has a… 9 Hugging Face Daily Papers research 1mo ago DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection Abstract DICS selects high-quality visual instruction data by measuring internal sample consistency, improving model performance with far less training data. Generated by thinkingmachines/Inkling-Small Visual instruction tuning is crucial for advancing the vision-language… 17 Vercel — AI dev-tools 1mo ago Compute that takes any shape Workloads differ in how long they run, how much memory they take, and how much of the environment they control. That previously meant a different compute primitive for each job, so product iterations required provisioning infrastructure, not just working on a feature. We built… 14 Hugging Face Daily Papers research 1mo ago Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactions Abstract MineAmongUs introduces a 3D multimodal Among Us environment and the ARIA harness to study embodied VLM-agent deception through verbal and non-verbal actions, revealing non-verbal channels as key to winning. Generated by thinkingmachines/Inkling-Small Strategic deception… 30 Hugging Face Daily Papers research 1mo ago CogEvol: Towards Efficient and Reliable Learning Environment Generation Abstract CogEvol is a family of models that generate structured learning artifacts in a single pass using supervised fine-tuning and reinforcement learning with vision-language rewards, achieving high quality with far fewer parameters and lower cost. Generated by… 6 Hugging Face Daily Papers research 1mo ago GenFirst: Generation Before Reconstruction for Stable End-to-End Latent Generative Modeling Abstract Direct end-to-end training of latent generative models avoids collapse via entropy preservation and asymmetric dynamics, using a generation-first strategy to achieve state-of-the-art image synthesis and unified multimodal generation. Generated by… 19 Hugging Face Daily Papers research 1mo ago CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents Abstract CAST improves LLM agent reliability by generating structured action-level rationales from sparse outcomes to train critique and policy models. Generated by thinkingmachines/Inkling-Small Large language model (LLM) agents are increasingly deployed in long-horizon,… 38 Hugging Face Daily Papers research 1mo ago DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution Abstract A compact 7B native joint audio-video generator uses cross-modal attention, progressive joint training, reinforcement learning with multimodal feedback, and an autoregressive 2K refinement pipeline to produce synchronized high-resolution outputs. Generated by… 30 arXiv — Machine Learning research 1mo ago HalluPrism: When Multimodal Uncertainty Should Diagnose, Not Decide arXiv:2608.29193v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) can assign similar confidence to answers that fail for different reasons. We propose HalluPrism, a behavioral diagnostic that re-runs an answer after visual degradation, blank-image… 23 arXiv — Machine Learning research 1mo ago MEL: Coordinate-Preserving EEG Tokenization for fMRI Translation arXiv:2608.29304v1 Announce Type: new Abstract: Translating electroencephalography (EEG) into functional magnetic resonance imaging (fMRI) is important for medical neuroimaging, clinical brain-state monitoring, and multimodal neural decoding, because it aims to infer spatially… 34 arXiv — Machine Learning research 1mo ago Behavioral Latency as Weak Event-Time Supervision for EEG Reaction-Time Decoding arXiv:2608.29428v1 Announce Type: new Abstract: Single-trial EEG analyses are often organized around events and latencies, yet EEG-based reaction-time (RT) prediction is posed as scalar regression on a fixed stimulus-locked window. RT is treated as a window-level label rather… 24 arXiv — NLP / Computation & Language research 1mo ago Parametric Multimodal User Memory: Storing What Captions Cannot Carry arXiv:2608.28609v1 Announce Type: new Abstract: A personalized agent needs a user memory: a persistent model of who its user is. Today it is almost always text -- transcripts and captions retrieved by similarity. This serves the captionable half of a person ("my cat is named… 34 arXiv — NLP / Computation & Language research 1mo ago Looking Again: Measuring Sycophancy in the Reasoning Chains of Multimodal Models Under Pressure arXiv:2608.28623v1 Announce Type: new Abstract: Large multimodal reasoning models (LMRMs) are getting increasingly capable, primarily through generating explicit chain-of-thought reasoning before answering. In language models it has been observed that this performance often… 25 arXiv — NLP / Computation & Language research 1mo ago Do large language models scrutinise what they review? A multimodal audit of scoring calibration, error detection, and author-identity effects arXiv:2608.28626v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to generate peer reviews, prompting examination of their capacity for critical evaluation. This study evaluates two multimodal LLMs, Qwen2.5-VL-72B and Pixtral-Large-124B, as… 38 Page 10 of 10 · 500 articles ← Newer
