News / #multimodal Tag Multimodal 500 articles archived under #multimodal · RSS Sign in to follow Hugging Face Daily Papers research 6d ago Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs Abstract A framework quantizes vision-language models for mobile deployment using self-generated training data and a 2.7-bit format, compressing Llama 3.2 11B Vision Instruct to 3.7 GB with preserved visual question answering performance. Generated by… 17 arXiv — Machine Learning research 6d ago Asymmetric Capacity Allocation in Self-Refinement Pipelines arXiv:2608.21345v1 Announce Type: new Abstract: Self-refinement, typically structured as generation, critique, and revision, is a widely adopted paradigm for improving LLM generation and serves as a core mechanism in many LLM agents. While the three stages involve different… 37 arXiv — Machine Learning research 6d ago EviRank: Structured Relevance Evidence for Multimodal Image Re-ranking arXiv:2608.20886v1 Announce Type: cross Abstract: Real-world image search queries are multimodal and compositional: ``find this shirt in pink'' specifies an entity to retain, an attribute to modify, and context to ignore. Yet existing re-rankers either compress such multifaceted… 21 arXiv — NLP / Computation & Language research 6d ago COMET: Contrastive Motion-Enhanced Temporal Reasoning for Video Multimodal Large Language Models arXiv:2608.21030v1 Announce Type: cross Abstract: Video multimodal large language models have advanced significantly, yet fine-grained motion-temporal understanding remains fragile. The core bottleneck is not only sparse frame sampling, but also the lack of a complete temporal… 37 arXiv — Machine Learning research 6d ago Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs arXiv:2608.21134v1 Announce Type: cross Abstract: Deploying vision-language models (VLMs) on mobile devices is challenging due to their significant memory and compute requirements. We present a framework for quantizing VLMs for efficient inference on resource-constrained… 18 arXiv — NLP / Computation & Language research 6d ago The Divergence Hypothesis: Unmasking Lexical Interference and Label Bias in Mental Health NLP arXiv:2608.20353v1 Announce Type: new Abstract: Computational mental health (CMH) classifiers often degrade under distribution shift because human annotators and distant-supervision pipelines reward different linguistic signals. We introduce TSS (Triple-Stream Stress probe), a… 24 arXiv — NLP / Computation & Language research 6d ago Decoupled Vision-Language System for Multimodal Understanding and Generation arXiv:2608.20382v1 Announce Type: new Abstract: We introduce a new architecture design for multimodal large language models (MLLMs), Libra, capable of both multimodal understanding and generation. Libra architecture contains one vision system and one language system, connected… 23 arXiv — NLP / Computation & Language research 6d ago Ontology-Driven Structural Regularization for Document-Level Relation Extraction arXiv:2608.20856v1 Announce Type: new Abstract: Document-Level Relation Extraction (DocRE) relies heavily on costly manually annotated datasets, while large distant supervision resources such as DocRED distant remain underexploited due to noise. We show that a critical yet… 12 arXiv — NLP / Computation & Language research 6d ago Calibrating Criterion Revision in LLM Agents: Failure Modes and a Trace-Anchored Protocol arXiv:2608.20729v1 Announce Type: cross Abstract: Language-model agents can improve after failure or carry text across episodes without revising what counts as success. We study the narrower attribution problem of criterion revision: when criterion K0 accepts an outcome… 26 arXiv — NLP / Computation & Language research 6d ago GeoExplain: Multimodal Reasoning based on Hierarchy of Visual Information in Street View arXiv:2506.16633v3 Announce Type: replace Abstract: Multimodal reasoning is a process of understanding, integrating and inferring information across different data modalities. It has recently attracted surging academic attention. Although there are various tasks for evaluating… 24 Hugging Face Daily Papers research 6d ago Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs Abstract Hybrid-thinking multimodal language models suffer from response-pattern misalignment between thinking and non-thinking modes, which is addressed by a diagnostic benchmark and pattern-specific reinforcement learning penalties. Generated by thinkingmachines/Inkling-Small… 4 r/LocalLLaMA community 6d ago Qwen3.8-27B NVFP4 with vision + 451K token KV-cache on one RTX 5090 (power limited to 400W) at 120 tokens/s average Hello, So I've been trying lots of combinations in that never-ending landscape of options and settings. I wanted a proper quant of 3.8 27B running as fast as possible on my 5090 at 400W, with vision and with as much KV-cache as possible and with concurrency enabled (aiming at 3… 25 llama.cpp releases dev-tools 8d ago b10580 mtmd: support dots3-note vision+audio ( #27524 ) text: conversion init impl mtmd: conversion impl mtmd cpp Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co… 22 r/LocalLLaMA community 8d ago Strix Halo (8060S / gfx1151), Qwen-3.8-27B @ Q8 and Q6 UD v3, up to 256K ctx, llama.cpp, DFlash2, vision, real workloads quality and steady performances, optimized recipes, ... Hi fellows fully-local halos, after manually following existing guides, I decided to build an LLM API endpoint installation and optimization guide that works even when autonomously followed by my pi agent, so I can install/experiment/reinstall easily and without babysitting. Q8… 38 r/LocalLLaMA community 8d ago Qwen3.8-27B on an RTX 5060 Ti 16GB: IQ4 vs Q8, 64K context, MTP, vision, and agent benchmarks I’ve been testing Qwen3.8-27B as a possible replacement for the Qwen3.5-9B that I have been running on RTX 5060Ti 16G. The goal was not just maximum tokens/sec, but useful context capacity, reliable tool calling, multi-turn behavior, and vision on a single 16GB GPU for true… 34 Hugging Face Daily Papers research 8d ago The Embedder's Dilemma: LLMs Are Better, but at What Cost? Abstract Large language models and dedicated embedding models achieve nearly identical aggregate performance across diverse tasks, but embedding models are far cheaper and faster, supporting a division of labor by task type. Generated by thinkingmachines/Inkling-Small Should you… 33 Hugging Face Daily Papers research 8d ago τ_0-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation Abstract A hierarchical vision-language-action model improves long-horizon robot manipulation by using world-model-guided test-time search to scale computation for high-level subtask decisions. Generated by thinkingmachines/Inkling-Small Long-horizon robot manipulation requires… 5 r/LocalLLaMA community 8d ago DeepSeek Harness v0.1.1 released https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.1-rc.1 The DeepSeek adapter adds the multimodal visual understanding model DeepSeek-V4-Flash-Vision-Exp. It also supports configuring native image requests. Commands such as /goal and /plan can accept text and… 22 Hacker News — AI on Front Page community 9d ago DeepSeek-v4-flash-vision-exp Article URL: https://api-docs.deepseek.com/guides/vision/ Comments URL: https://news.ycombinator.com/item?id=49386163 Points: 286 # Comments: 87 18 r/LocalLLaMA community 9d ago DeepSeek-V4-Flash-Vision-Exp   submitted by   /u/Xhehab_ [link]   [comments] 8 r/MachineLearning community 9d ago Rejected at EMNLP with decent scores. What can be done next? [D] So I got rejected at EMNLP with scores:- Meta: 3 (very positive in the review) Reviewers: OA(conf) 3(4) 3(4) 2.5(3) Avg: 2.83(3.67) Track: multimodality Rebuttals never got any acknowledgements. Most weaknesses were already discussed in the paper. What are my options now? As it… 24 Hugging Face Daily Papers research 9d ago EXIMO: VLM Guided Exploration of VLA Policies Abstract EXIMO efficiently fine-tunes large vision-language-action robot policies by combining VLM-guided exploration, imitation on orchestrated data, and residual off-policy reinforcement learning. Generated by thinkingmachines/Inkling-Small How to efficiently finetune robot… 6 Smol AI News news-outlet 9d ago not much happened today **Ox Alpha** emerged as a mystery model with strong coding and agentic performance, likely a **Zhipu/GLM-family** model such as **GLM-5.3 Vision**. Analysts suggest its gains come from post-training and infrastructure improvements rather than sheer size, based on the **743B… 10 arXiv — Machine Learning research 9d ago Holtercare-Bench: A Multimodal Benchmark for Evaluating Long-Term Dynamic ECG Analysis arXiv:2608.19297v1 Announce Type: new Abstract: While multimodal large language models (MLLMs) excel in medical applications, most of them favor static images or short-term signals. In the critical field of dynamic electrocardiograms (ECG), models struggle with complex temporal… 19 arXiv — Machine Learning research 9d ago Beyond Multimodal Alignment: Certifying Physical Language through Response Substitution and Ordered Execution arXiv:2608.19492v1 Announce Type: new Abstract: World models increasingly treat compact multimodal representations as interfaces between perception and physical interaction, yet existing probes do not establish whether different sensors carry the same executable meaning or… 20 arXiv — NLP / Computation & Language research 9d ago MileGPO: Milestone Inference with Local Evidence for Graph-Based Policy Optimization of Long-Horizon LLM Agents arXiv:2608.19803v1 Announce Type: cross Abstract: Credit assignment is challenging in long-horizon agentic reinforcement learning, where supervision often comes only from final rewards. Existing methods refine trajectory-level signals into step-level credits through step… 30 arXiv — Machine Learning research 9d ago Answer-Level Trust Selection for Physical Vision-Language Reasoning arXiv:2608.19807v1 Announce Type: new Abstract: Vision-language models (VLMs) can estimate physical quantities such as duration, speed, and acceleration from visual observations, but existing benchmarks primarily assess overall model performance against annotated ground truth.… 4 arXiv — Machine Learning research 9d ago Auditing Recorded Predictive Lead Service-Line Classifications Against Physical Verification: A Statewide Study of New York arXiv:2608.19922v1 Announce Type: new Abstract: Under the US Lead and Copper Rule Revisions, a utility may determine a service line's material with a predictive model instead of inspecting it. New York State publishes, per address, which method was used. Almost no address… 19 arXiv — NLP / Computation & Language research 9d ago Time-Series Retrieval for Grounding Multimodal Language Models in Remaining Useful Life arXiv:2608.19218v1 Announce Type: new Abstract: Large language models (LLMs) and agentic AI systems are increasingly being explored for domain-specific maintenance and prognostics tasks, raising the question of whether they can effectively support prognostics and health… 31 arXiv — Machine Learning research 9d ago Clustering and Token Denoising for Faster and More Robust VLMs arXiv:2608.19285v1 Announce Type: cross Abstract: Recent Visual-Language Models (VLMs) have enhanced the capabilities of pre-trained LLMs by adding vision tokens alongside text, with approaches like LLaVA showing impressive results. However, the computational burden of… 29 arXiv — Machine Learning research 9d ago Concentrated Liquidity Provision: a Reinforcement Learning Perspective arXiv:2608.19389v1 Announce Type: cross Abstract: Automated market makers (AMMs) are a cornerstone of decentralised finance (DeFi). Constant product markets with concentrated liquidity, such as UniswapV3, are now a well-established design. In these markets, liquidity providers… 8 arXiv — Machine Learning research 9d ago Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress arXiv:2608.19408v1 Announce Type: cross Abstract: On-policy distillation (OPD) has emerged as an effective framework for post-training language models by pairing student-generated trajectories with dense token-level supervision from a teacher. However, OPD implicitly assumes… 24 arXiv — Machine Learning research 9d ago Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulation arXiv:2608.19490v1 Announce Type: cross Abstract: State-of-the-art vision-language-action (VLA) models such as $\pi_{0.5}$ exhibit strong semantic understanding, instruction following and task behavior. However, when deployed on new robots, even minor mismatches in hardware… 5 arXiv — Machine Learning research 9d ago Scaffolding Minds: Optimizing Latent Visual Target Representations for Multimodal Reasoning arXiv:2608.19669v1 Announce Type: cross Abstract: Latent reasoning has advanced multimodal reasoning through a two-stage training paradigm: (1) a helper image is encoded into latent tokens to teach visual chain-of-thought during a supervised fine-tuning (SFT) stage, and (2)… 24 arXiv — Machine Learning research 9d ago RIPE++: Reinforced Keypoint Learning from Positive Pairs Only arXiv:2608.19693v1 Announce Type: cross Abstract: Sparse keypoint extraction and matching underpin core tasks in geometric computer vision, including structure-from-motion, visual SLAM, augmented reality, and medical image registration. Learning robust local feature… 32 arXiv — NLP / Computation & Language research 9d ago Projector Is All You Train arXiv:2608.19726v1 Announce Type: new Abstract: The typical training process of a multimodal large language model (MLLM) involves adapting both the language model backbone and the projector between the backbone and a modality-specific encoder. We ask whether fine-tuning the… 9 arXiv — Machine Learning research 9d ago Question-Guided Evidence Acquisition for Multimodal Visual Question Answering arXiv:2608.19739v1 Announce Type: cross Abstract: Multimodal LLMs can see a document, but they often can't read it reliably. Small text, tables, visual cues, and topological elements still trip them up under direct visual inference, even when the page is already sitting in the… 14 arXiv — NLP / Computation & Language research 9d ago Compliance, Capability, and Conflict: Benchmarking Multimodal LLMs under System Messages arXiv:2608.19207v1 Announce Type: new Abstract: Production deployments of Multimodal Large Language Models (MLLMs) increasingly rely on system messages to govern model behavior. Yet existing benchmarks either evaluate constraints in text only or embed them into the user turn,… 30 arXiv — NLP / Computation & Language research 9d ago When Irrelevant Text Matters: Affine Margin Shifts in Multimodal Large Language Models arXiv:2608.19208v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are frequently exposed to auxiliary textual context, the impact of which on visually grounded tasks remains underexplored. In this paper, we investigate the influence of task-irrelevant… 31 arXiv — NLP / Computation & Language research 9d ago NepOOC-M: Bilingual Nepali-English Benchmark and Comparative Analysis of Multimodal Architectures for OOC Detection arXiv:2608.19212v1 Announce Type: new Abstract: Out-of-context (OOC) misinformation pairs authentic images with misleading captions to construct false narratives without image manipulation, making detection a problem of multimodal alignment rather than image forensics. Despite… 20 arXiv — NLP / Computation & Language research 9d ago Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization for Multimodal Sarcasm Detection arXiv:2608.19942v1 Announce Type: new Abstract: Multimodal sarcasm detection aims to identify sarcastic intent from multimodal content, where inconsistencies between literal meaning and contextual cues often signal irony. This task has attracted increasing research attention.… 31 arXiv — NLP / Computation & Language research 9d ago Robust Incomplete Multimodal Sentiment Analysis via Iterative Proxy Correction arXiv:2608.19971v1 Announce Type: new Abstract: Multimodal sentiment analysis aims to infer affective states by integrating language, visual, and acoustic cues. However, real-world multimodal inputs are often incomplete or corrupted, which can weaken cross-modal complementarity… 30 arXiv — NLP / Computation & Language research 9d ago Does Listening Matter? Backchanneling and Nodding in AI Clone arXiv:2608.19527v1 Announce Type: cross Abstract: AI clones that imitate a specific person typically reproduce what the person says and how they sound, but not how they listen. We investigate whether adding multimodal listening behaviors gives such a clone more presence and… 14 arXiv — NLP / Computation & Language research 9d ago PEA-DPO: Perception-Enhanced Alignment Direct Preference Optimization for MLLMs Alignment arXiv:2608.19598v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) has emerged as an effective approach for aligning large language models (LLMs) with human preferences. However, its adaptation to multimodal settings remains unexplored. Through… 9 arXiv — NLP / Computation & Language research 9d ago TempJail: Temporal Jailbreak Attack against Large Vision-Language Models via Subtitle Scheduling arXiv:2608.19737v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) have achieved remarkable progress in video understanding and reasoning. Despite extensive studies on text- and image-based jailbreaks, video jailbreaks against LVLMs remain largely unexplored.… 19 arXiv — NLP / Computation & Language research 9d ago Towards Clinically Faithful Medical Image Captioning via Enhanced Vision-Language Alignment arXiv:2608.19825v1 Announce Type: cross Abstract: Medical image captioning is a technique that accelerates early-stage diagnostic workflows and enhances the interpretability of medical diagnostic AI systems. However, unlike general image captioning, clinically reliable… 26 arXiv — NLP / Computation & Language research 9d ago Explainable Multimodal Depression Recognition in Clinical Interviews via PHQ-Aligned Symptom Summarization arXiv:2501.16106v2 Announce Type: replace Abstract: Recent advances in multimodal depression recognition for clinical interviews (MDRC) have demonstrated the potential of AI systems by integrating textual, acoustic, and facial cues. However, existing methods pay limited… 35 arXiv — NLP / Computation & Language research 9d ago SPyCE: Skill-Policy Co-evolution for Multimodal Agents arXiv:2607.13854v2 Announce Type: replace Abstract: Multimodal agents that think with images iteratively manipulate visual evidence and invoke tools across many steps. Existing reinforcement learning methods reduce trajectories to scalar rewards, forcing the policy to discover… 33 Vercel — AI dev-tools 9d ago DeepSeek V4 Flash Vision Experimental now available on AI Gateway DeepSeek V4 Flash with vision is now available on AI Gateway. This model is an experimental version that accepts images alongside text. You can ask it to describe a picture, read text out of a screenshot, or work through a chart in the same request as your prompt. DeepSeek V4… 13 Hugging Face Daily Papers research 10d ago Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL Abstract Co-RL enables unsupervised reasoning via cooperative multi-agent reinforcement learning with peer-derived rewards, improving performance across text and vision tasks without ground-truth labels. Generated by thinkingmachines/Inkling-Small Reinforcement learning (RL) has… 36 Page 3 of 10 · 500 articles ← Newer Older →
