News / #multimodal Tag Multimodal 500 articles archived under #multimodal · RSS Sign in to follow arXiv — Machine Learning research 25d ago Learning Molecular Representations from Cellular Phenotypes with Structure Preservation arXiv:2608.02688v1 Announce Type: new Abstract: Phenotypic drug discovery enables the discovery of functional relationships between molecular structures and cellular responses. However, existing multimodal representation learning methods often optimize cross-modal alignment… 11 arXiv — Machine Learning research 25d ago PatTree: a novel approach for automated creation of multimodal, graph-based patient representations for medical classification tasks arXiv:2608.02692v1 Announce Type: new Abstract: Access to holistic, multimodal data improves the performance of Artificial Intelligence (AI) in medical classification tasks compared to utilizing single modalities or data sources. However, the inherent heterogeneity and… 24 arXiv — Machine Learning research 25d ago Bayesian Data Reweighting Improves Multimodal Retrieval for Knowledge-Based Visual Question Answering arXiv:2608.02907v1 Announce Type: new Abstract: Multimodal retrievers are essential for knowledge-based visual question answering, where they retrieve external evidence for image-question pairs. However, existing contrastive training methods typically treat all unmatched… 4 arXiv — Machine Learning research 25d ago CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning arXiv:2608.03673v1 Announce Type: new Abstract: Many critical reasoning tasks, including clinical diagnosis, legal judgment, and industrial fault diagnosis, require step-dependent causal chains in which early errors propagate and correct conclusions can mask invalid reasoning.… 28 arXiv — Machine Learning research 25d ago Enhancing VLM Reward Models Through Structure-Aware Fine-Tuning arXiv:2608.03875v1 Announce Type: new Abstract: Designing effective reward functions remains a major bottleneck in Reinforcement Learning (RL). Recent work uses large foundation Vision-Language Models (VLMs) as reward models, computing text-observation similarity to bypass… 8 arXiv — NLP / Computation & Language research 25d ago OncoTriad-QA: A Patient-Level Radiology-Pathology-Genomics Benchmark for Pan-Cancer Reasoning arXiv:2608.02615v1 Announce Type: new Abstract: Cancer diagnosis and characterization require integrating complementary evidence from radiology, pathology, genomics, and clinical metadata. However, most medical large language model (LLM) and vision-language model (VLM)… 10 arXiv — NLP / Computation & Language research 25d ago Aligning Large Vision-Language Models at Test Time: A Trajectory-Guided Structured Sampling Approach arXiv:2608.03204v1 Announce Type: new Abstract: Post-training reinforcement learning (RL) algorithms are commonly used to align large vision-language models (LVLMs) with human intent and the requirements of visual reasoning tasks. However, existing RL-based alignment methods are… 18 arXiv — NLP / Computation & Language research 25d ago ArtECulture: Benchmarking Culture-Conditioned Visual Emotion Understanding in Multimodal Large Language Models arXiv:2608.03358v1 Announce Type: new Abstract: Existing visual emotion understanding methods typically ignore cultural variations in emotional perception. We introduce culture-conditioned visual emotion understanding, a task that predicts the culture-specific emotional… 10 arXiv — NLP / Computation & Language research 25d ago TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning arXiv:2608.04007v1 Announce Type: new Abstract: Tool-Integrated Reasoning (TIR) enables LLMs to solve complex tasks through iterative tool interactions. However, existing reinforcement learning methods often rely on trajectory-level supervision, limiting fine-grained credit… 24 arXiv — NLP / Computation & Language research 25d ago CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning arXiv:2608.02833v1 Announce Type: cross Abstract: Chart question answering (CQA) requires multimodal large language models (MLLMs) to integrate visual comprehension with logical reasoning, yet current models struggle with accurate visual grounding and coherent reasoning chains.… 14 Hugging Face Daily Papers research 25d ago Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editing Abstract Recent advances in image generation have demonstrated the potential of unified multimodal models that integrate understanding, generation, and editing. However, unified 3D modeling remains constrained by scarce multimodal data, particularly the lack of large-scale and… 8 Hugging Face Daily Papers research 25d ago CAPEval: A Decoupled Caption Evaluation across Understanding and Generation Abstract Captions serve as a primary supervision signal for both multimodal understanding and text-to-image generation. However, previous evaluations treat the caption quality as a single scalar objective, which conflates two distinct properties: (1) how much visual information… 22 Hugging Face Daily Papers research 25d ago TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning Abstract Tool-Integrated Reasoning (TIR) enables LLMs to solve complex tasks through iterative tool interactions. However, existing reinforcement learning methods often rely on trajectory-level supervision, limiting fine-grained credit assignment in long-horizon TIR scenarios.… 26 Hugging Face Daily Papers research 26d ago Wnuan: Staged Post-Training for Question Answering over Proprietary Enterprise Knowledge Abstract Enterprise question answering requires models to acquire proprietary knowledge without discarding general capabilities. We present Wnuan, a three-stage pipeline that constructs task-oriented supervision from documents, performs supervised fine-tuning with general-data… 24 Hacker News — AI on Front Page community 26d ago Mistral's Shieldstral: 3B open-weights model for multimodal moderation Article URL: https://mistral.ai/news/shieldstral/ Comments URL: https://news.ycombinator.com/item?id=49171268 Points: 307 # Comments: 74 7 Hugging Face Daily Papers research 26d ago Loud or Silent? A Reusable Framework for Per-Modality Failure Analysis in Multimodal Clinical AI Abstract Multimodal clinical models are usually judged on accuracy with every modality present, but deployment removes modalities; an echocardiogram is often unavailable where an ECG is routine. Two questions then matter beyond the size of the accuracy loss: which modality was… 14 Hugging Face Daily Papers research 26d ago Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models Abstract Multimodal Large Language Models (MLLMs) achieve strong performance by integrating visual inputs with the rich priors of pretrained language models. However, they often fail on vision-centric tasks, especially when visual evidence conflicts with pretrained knowledge. We… 26 Hugging Face Daily Papers research 26d ago ICDAR 2026 Competition on Information Extraction from Atomic Layer Deposition/Etching (ALD/E) Scientific Figures Abstract Scientific figure comprehension and reasoning using multimodal AI requires integrating visual perception with domain-specific reasoning to extract meaningful knowledge, often not presented in the text of a research publication. The Sci-ImageMiner benchmark dataset,… 6 r/LocalLLaMA community 26d ago Why are Chinese models better* at Frontend than the western top labs? I use A LOT both openAI and Anthropic products. When I need some frontend work (pure web dev) (or answer that feel less verbose and more to the point) I use Anthropic. For multimodality openAI feels better (understanding audio, screenshots, generating images, etc). But openAI… 17 Hugging Face Daily Papers research 26d ago Relax Within, Balance Across: Geometry-Guided Load Balancing for Vision-Language Mixture-of-Experts Abstract Vision-language MoE batches contain different numbers of image and text tokens. Image resolution, image count, tiling, and prompt length all change this token mix. We call the standard token-level Switch auxiliary loss Std-Aux. Std-Aux balances only the mixed load, so… 33 Hugging Face Daily Papers research 26d ago DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agents Abstract Multimodal large language models (MLLMs) have advanced visual understanding and reasoning, yet their static parametric knowledge limits their ability to address knowledge-intensive and dynamically evolving open-world problems. To move beyond this limitation, multimodal… 24 Hugging Face Daily Papers research 26d ago 3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering Abstract Recent 3D vision-language models (3D VLMs) construct geometry aware tokens by projecting 2D visual features into world coordinates, enabling spatial reasoning for tasks such as 3D question answering. However, this design generates thousands of tokens per scene,… 13 Hugging Face Daily Papers research 26d ago LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks Abstract Large language model (LLM) agents increasingly undertake long-horizon tasks that require sustained reasoning, tool use, and revision across many interdependent steps. However, existing agent harnesses maintain task execution, task state, and completion assessment within… 37 Smol AI News news-outlet 26d ago not much happened today **Alibaba** launched **Qwen3.8-Max**, enhancing multimodal capabilities and agent ecosystem integration. **NVIDIA** introduced **Alpamayo 2 Super** for autonomous vehicle reasoning, while **Mistral AI** released **Shieldstral**, a 3B parameter open-weights safety model for… 17 arXiv — Machine Learning research 26d ago Rethinking Pretraining for Specialized Design Data: Evidence from the JONES-19 Cultural Design Dataset arXiv:2608.00135v1 Announce Type: new Abstract: Design and architectural archives encode expert human knowledge in graphical formats, providing a critical testbed for design-inspired Machine Learning (ML) challenges absent with typical computer vision benchmarks. Building on… 33 arXiv — Machine Learning research 26d ago RHEA: Reliability-Harmonized Reconstruction and Assignment for Robust Multimodal-Attributed Graph Clustering arXiv:2608.00621v1 Announce Type: new Abstract: Multimodal-attributed graphs (MAGs), whose nodes carry heterogeneous attributes such as text and images over a relational structure, have become a fundamental substrate for label-free entity grouping tasks, including community… 23 arXiv — Machine Learning research 26d ago Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity arXiv:2608.00623v1 Announce Type: new Abstract: Multimodal-attributed graphs (MAGs), where nodes carry heterogeneous semantic content across multiple modalities while edges encode relational dependencies, have been widely adopted across diverse domains. Federated multimodal… 4 arXiv — Machine Learning research 26d ago Generic Vision and Cross-Attention for Reaction Yield Prediction arXiv:2608.00776v1 Announce Type: new Abstract: Traditional reaction yield prediction is constrained by 1D quantum descriptors that lack explicit spatial information. To address this gap, a dual-modal Vision Cross-Attention architecture is proposed, fusing tabular… 36 arXiv — Machine Learning research 26d ago GeoArbiter: Verifiability-Guided Grounding for Remote-Sensing Multimodal LLMs arXiv:2608.00877v1 Announce Type: new Abstract: Remote-sensing multimodal large language models (MLLMs) often assert facts that imagery cannot establish, such as a facility's identity or function. Coordinate-keyed geographic retrieval can supply this missing knowledge, improving… 28 arXiv — Machine Learning research 26d ago Caliber: Cross-Architecture Extraction-Cost Control for Score-Returning APIs arXiv:2608.01023v1 Announce Type: new Abstract: We present Caliber, an output-perturbation defense against model extraction that formulates noise selection as a calibration problem: how much the defense degrades the supervision signal used to train a surrogate, and the provable… 6 arXiv — Machine Learning research 26d ago Distill What the Student Can See: Fisher-Projected On-Policy Distillation for Vision-Language Models arXiv:2608.01263v1 Announce Type: new Abstract: On-policy distillation (OPD) samples trajectories from the current student policy and minimizes token-level divergence between student and teacher next-token distributions at prefixes along those trajectories. This aligns the… 6 arXiv — NLP / Computation & Language research 26d ago Obshazard-bench: Benchmarking Multimodal Foundation Models for Real-Time Disaster Intelligence from Raw Earth Observation Streams arXiv:2608.00012v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are increasingly used to interpret Earth observation data, yet their capability to support real-world disaster emergency response remains insufficiently evaluated. Existing remote sensing… 26 arXiv — NLP / Computation & Language research 26d ago What Transfers from Text to Vision? Capability Scaling Laws and Transfer Dynamics for VLMs arXiv:2608.00013v1 Announce Type: new Abstract: Choosing the right large language model (LLM) backbone is the most consequential decision when building a vision-language model (VLM), yet it remains fundamentally unprincipled: compute-based scaling laws fail to generalize across… 11 arXiv — NLP / Computation & Language research 26d ago Cross-Task Dissociation in Frontier Vision-Language Model Theory of Mind arXiv:2608.00261v1 Announce Type: new Abstract: Do frontier vision-language models present a coherent Theory-of-Mind (ToM) profile across tasks, matching the same human reference group, or does that profile fragment from one paradigm to the next? We evaluate a shared panel of… 22 arXiv — NLP / Computation & Language research 26d ago AdaMTP: An Adaptive Training Paradigm for Multi-Token Prediction arXiv:2608.00434v1 Announce Type: new Abstract: Multi-Token Prediction (MTP) has emerged as an effective paradigm that augments a shared Large Language Model backbone with auxiliary heads, training the model to predict several future tokens in parallel to enrich its supervision… 28 arXiv — NLP / Computation & Language research 26d ago Evaluating VLMs on Multimodal Aristotelian Persuasion Tasks arXiv:2608.01238v1 Announce Type: new Abstract: Vision Language Models (VLMs) have demonstrated exceptional performance across various tasks. However, they have not yet been thoroughly evaluated on more complex tasks. The Persuasion Model, conceived by Aristotle, resembles a… 35 arXiv — NLP / Computation & Language research 26d ago LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning arXiv:2608.01328v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are rapidly evolving with expanded context windows and stronger reasoning capabilities, enabling multi-chart understanding and multi-step inference. These abilities are increasingly… 10 arXiv — NLP / Computation & Language research 26d ago Discriminative Axis, Not Data Volume: What a Contrastive Corpus Teaches an Audio Embedding arXiv:2608.01560v1 Announce Type: new Abstract: Scaling the corpus is the default remedy when a contrastive representation lacks an attribute. We report a case where it does nothing, and identify what does: adding a lexical-speech round to a frozen-base multimodal embedding… 29 Hugging Face Daily Papers research 26d ago UEmbed: Unified Sparse and Dense Multimodal Embeddings Abstract Sparse retrieval underpins modern search systems, from web search to retrieval-augmented generation. Existing work has introduced Learned Sparse Retrieval (LSR) to push beyond exact lexical matching toward richer semantics. Yet LSR has so far remained tied to… 5 Hugging Face Daily Papers research 26d ago WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning Abstract Reinforcement learning (RL) post-training of Vision-Language-Action (VLA) models has shown strong promise for robotic manipulation. Among RL methods, critic-based approaches rely on a value estimator that predominantly operates on single-frame observations or… 24 Hugging Face Daily Papers research 26d ago Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs Abstract Recent Vision-Language-Action (VLA) models for autonomous driving (AD) increasingly utilize chain-of-thought (CoT) supervision to enhance the reasoning capabilities of their Vision-Language Model (VLM) components, yet existing annotation pipelines commonly expose the… 28 Hugging Face Daily Papers research 26d ago VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation Abstract Multimodal on-policy distillation (OPD) transfers fine-grained visual knowledge by supervising student-generated trajectories with a privileged-view teacher. Yet its next-token corrections are source-mixed, combining visual signals with linguistic priors and… 33 r/MachineLearning community 26d ago I created an autonomous boxing benchmark [D] I created an AI boxing match to test the decision speed, adaptability and strategy. I fed the LLMs with data about the current match and if they have vision, they will get even more data. The match has street rules, anything goes and an AI is not defeated until the ref counts to… 24 Hugging Face Daily Papers research 27d ago N_0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation Abstract We present N_0-TWAM, a tactile-native world-action model for contact-rich manipulation that predicts both future vision and future contact. To our knowledge, it is the first tactile world-action model trained at large scale, and it shows strong capability on… 31 Smol AI News news-outlet 27d ago Qwen 3.8 Max **Alibaba** launched **Qwen3.8-Max**, a **2.4T-parameter** open-weight model emphasizing autonomous coding, long-horizon execution, and multimodal feedback, with aggressive pricing. Early benchmarks rank it highly on human-preference and vision tasks, showing parity with… 21 arXiv — Machine Learning research 27d ago Mitigating Class-Tail Undercoverage in Medical Vision-Language Models under Clinical Shift arXiv:2607.28696v1 Announce Type: new Abstract: Medical vision-language models (VLMs) can retain high observed marginal coverage after clinical shift while substantially under-covering an individual disease class. The affected class varies with acquisition protocol and backbone… 9 arXiv — Machine Learning research 27d ago MMFGU: Multimodal Federated Graph Unlearning arXiv:2607.28708v1 Announce Type: new Abstract: Multimodal federated graph learning enables clients to collaboratively train graph models over structural, textual, and visual signals without sharing private local data. However, the presence of heterogeneous multimodal content… 32 arXiv — Machine Learning research 27d ago Reflection or Re-Generation? Why LLM Revision Fails Where Human Revision Succeeds arXiv:2607.28908v1 Announce Type: new Abstract: Reflection, the ability to revisit and revise prior reasoning, is central to how humans improve their answers. Large language models (LLMs) are increasingly prompted to "reflect," yet whether this resembles human revision remains… 34 arXiv — Machine Learning research 27d ago Adaptive FastOPD: Progress-Aware Rollout Horizon Expansion for Efficient On-Policy Distillation arXiv:2607.29494v1 Announce Type: new Abstract: On-policy distillation (OPD) provides dense teacher supervision along student-generated trajectories, but its online rollout process incurs substantial computational cost, particularly when a few long responses delay batch… 6 arXiv — Machine Learning research 27d ago DeltaServe: Host-Agnostic Co-Serving of Inference and Fine-Tuning for LLMs arXiv:2607.28848v1 Announce Type: cross Abstract: LLM serving systems are provisioned for peak load to meet strict latency targets, leaving substantial GPU compute idle whenever traffic falls below peak. We present DeltaServe, a host-agnostic co-serving design that converts this… 20 Page 9 of 10 · 500 articles ← Newer Older →
