News / #multimodal Tag Multimodal 500 articles archived under #multimodal · RSS Sign in to follow arXiv — NLP / Computation & Language research 6d ago CARE: Certifying Acceleration for Vision-Language-Action Inference arXiv:2610.08917v1 Announce Type: new Abstract: While vision-language-action (VLA) models have advanced rapidly, running them at every control step remains expensive. Prior work accelerates VLA inference using techniques like action chunking and visual-token pruning, typically… 15 arXiv — NLP / Computation & Language research 6d ago SpikingVLA: Asynchronous Spiking Vision-Language-Action Models arXiv:2610.09710v1 Announce Type: new Abstract: ANN-to-SNN conversion offers a practical route toward energy-efficient spiking Vision-Language-Action (VLA) models by bypassing the substantial cost of training large-scale SNNs from scratch. However, existing methods often require… 5 arXiv — NLP / Computation & Language research 6d ago InterView-C: A Synchronized Multimodal Corpus of VR Avatar-Mediated Survey Interviews arXiv:2610.10145v1 Announce Type: new Abstract: We present InterView-C, a German multimodal corpus of 27 survey interviews conducted entirely in virtual reality, with both interlocutors represented by avatars. The corpus aligns spoken interaction with synchronized behavioral… 29 arXiv — NLP / Computation & Language research 6d ago Learning to Act with Task Progress: Distilling Small Agents from Compact Teacher Supervision arXiv:2610.10332v1 Announce Type: new Abstract: Learning from large-model demonstrations offers a way to train small agents that can complete recurring tasks without calling a large model at every step. A central design choice is what to retain from teacher trajectories that… 23 arXiv — NLP / Computation & Language research 6d ago Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution arXiv:2610.09240v1 Announce Type: cross Abstract: Modern web agents built on large vision-language models process webpages, select relevant UI elements, and translate model outputs into browser actions. Existing visual red-teaming approaches use adversarial visual content to… 14 arXiv — NLP / Computation & Language research 6d ago TopoGraphRAG-Bench: Evaluating Multimodal GraphRAG on Layout-Grounded Evidence Reasoning arXiv:2610.09360v1 Announce Type: cross Abstract: Real-world documents distribute evidence across text, tables, figures, and captions within complex page layouts. Answering complex questions over such documents therefore requires more than retrieving relevant passages: systems… 33 The Information — AI news-outlet 6d ago Amazon Lays Off Hundreds in Stores Division Amazon laid off hundreds of people in its retail division today, a spokesperson confirmed. “We’ve adjusted parts of our Stores business because we believe this structure will better enable us to deliver on our priorities,” a spokesperson said, adding Amazon is always looking at… 10 llama.cpp releases dev-tools 7d ago b11481: mtmd: add cohere2 vision support (#30062) cohere2 vision model address comments remove redundant mapping follow existing patterns fused linear_1 31 Hugging Face official-blog 7d ago Multimodal open d1 decision models for the edge Back to Articles a]:hidden"> Multimodal open d1 decision models for the edge Team Article Published October 7, 2026 Upvote - Aurelien Lac Aurelien-Lac LiquidAI Fernando Fernandes Neto fernandofernandes LiquidAI Edoardo Mosca EdoardoMosca LiquidAI Maxime Labonne mlabonne LiquidAI… 12 r/LocalLLaMA community 7d ago Image-text retrieval with EmbeddingGemma 2's vision tower, running in the browser on WebGPU EmbeddingGemma 2 came out this week. It maps images and text into one 768-dim space, so you can search photos by describing them. I ported its text and vision towers to ruNNtime, a WebGPU inference library in TypeScript, and made a small photo gallery where search runs entirely… 28 arXiv — Machine Learning research 7d ago EVFormer: An Egocentric Vision-EMG Bidirectional Attention Model for Bimanual Hand Pose Estimation arXiv:2610.06970v1 Announce Type: new Abstract: Egocentric bimanual hand pose estimation is important for virtual interaction, wearable control, and rehabilitation, but visual observations are often degraded by self-occlusion, hand-hand contact, and object manipulation. We… 15 arXiv — Machine Learning research 7d ago STOCK-JEPA: Prior-Anchored Latent Revision Representation Learning in Equity Markets arXiv:2610.07006v1 Announce Type: new Abstract: Learning effective representations helps characterize the structure and dynamics of equity markets from financial data with a low signal-to-noise ratio. Black-box deep models can capture complex patterns but may overfit sample… 24 arXiv — Machine Learning research 7d ago RELACE: retrospective likelihood-based action credit estimation for long-horizon language agents arXiv:2610.07349v1 Announce Type: new Abstract: Group Relative Policy Optimization (GRPO) avoids a separate critic by estimating advantages from rollout groups. For multi-turn agents, however, trajectory-level supervision provides coarse, noisy credit: terminal rewards do not… 38 arXiv — Machine Learning research 7d ago Efficient Multimodal Inference through Adaptive Acquisition and Sequential Fusion arXiv:2610.07466v1 Announce Type: new Abstract: Multimodal systems often encode every available input, even when a subset suffices for prediction. Adaptive acquisition can reduce this cost by using predictions from incrementally fused evidence to decide which modality to encode… 6 arXiv — Machine Learning research 7d ago Source-Learned Reliance for Selective Test-Time Adaptation of Multimodal Time Series arXiv:2610.07499v1 Announce Type: new Abstract: Multimodal wearable systems must remain reliable when sensor streams become noisy or unavailable. Existing multimodal test-time adaptation (TTA) methods often assess reliability online, but cross-modal agreement can be misleading… 19 arXiv — NLP / Computation & Language research 7d ago EMODE: Dynamic Para-Semantic Experts for Emotion-Aware Speech Language Modeling arXiv:2610.06956v1 Announce Type: new Abstract: Large speech language models have demonstrated strong capabilities in unified cross-modal understanding and generation, yet paralinguistic cues, especially emotion, remain difficult to preserve. Existing systems typically rely on… 14 arXiv — NLP / Computation & Language research 7d ago Two Vectors Replace In-Context Demos: Structured Task Adaptation via Embeddings arXiv:2610.07572v1 Announce Type: new Abstract: In-context learning (ICL) adapts frozen large multimodal models (LMMs) to new tasks from a few demonstrations (demos), but re-encodes them at every query, where each demo image adds up to hundreds of visual tokens. Demo-free… 14 arXiv — NLP / Computation & Language research 7d ago Reading, Not Manipulating: Leveraging Router Logits for Multimodal Safety in MoE Vision-Language Models arXiv:2610.07774v1 Announce Type: new Abstract: Vision-language models (VLMs) face compositional safety risks where harmful intent emerges from the interaction between visual and textual inputs. As mixture-of-experts (MoE) VLMs become increasingly common, recent work has… 9 arXiv — NLP / Computation & Language research 7d ago Visual Abstention in Unified Multimodal Models arXiv:2610.07887v1 Announce Type: new Abstract: Unified multimodal models (UMMs) integrate understanding and generation, yet their generative behavior is rarely governed by what they understand about the task. We formalize visual abstention: when a requested visual… 17 arXiv — NLP / Computation & Language research 7d ago Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability arXiv:2610.08448v1 Announce Type: new Abstract: On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student predictions requires alignment at both sequence and vocabulary levels. In this… 26 The Information — AI news-outlet 7d ago Pinterest Names New CFO Pinterest named Amazon executive James Dibbo as chief financial officer. It’s the fourth C-suite shakeup for the social media firm this year. Dibbo, who will start at Pinterest on October 26, previously led finance at Amazon’s media and retail divisions, most recently serving as… 38 Google DeepMind official-blog 8d ago EmbeddingGemma 2: an open, lightweight multimodal embedding model EmbeddingGemma 2: an open, lightweight multimodal embedding model Oct 06, 2026 | x.com Facebook LinkedIn Mail EmbeddingGemma 2 is the most capable model for on-device multimodal embeddings, natively mapping combinations of text, images, audio, and video into a unified embedding… 4 r/LocalLLaMA community 8d ago EmbeddingGemma 2 is a best-in-class open model for natively multimodal embeddings   submitted by   /u/sn2006gy [link]   [comments] 10 r/LocalLLaMA community 8d ago Introducing EmbeddingGemma 2: A best-in-class open model for natively multimodal embeddings | Google https://huggingface.co/google/embeddinggemma-2 EmbeddingGemma 2 is an open multimodal embedding model built by Google DeepMind which maps text (incl. code), images, video, and audio inputs—and combinations thereof—into a single, unified 768-dimensional vector space. The model… 9 llama.cpp releases dev-tools 8d ago b11452 model: support embeddinggemma2 (text+vision+audio) ( #30054 ) 4 Hacker News — AI on Front Page community 8d ago EmbeddingGemma 2: An open, lightweight multimodal embedding model Article URL: https://blog.google/innovation-and-ai/technology/developers-tools/embeddinggemma-2/ Comments URL: https://news.ycombinator.com/item?id=49980487 Points: 301 # Comments: 31 31 r/LocalLLaMA community 8d ago google/embeddinggemma-2 · Hugging Face EmbeddingGemma 2 is an open multimodal embedding model built by Google DeepMind which maps text (incl. code), images, video, and audio inputs—and combinations thereof—into a single, unified 768-dimensional vector space. The model has 740M total parameters, combining a 270M… 18 TechCrunch — AI news-outlet 8d ago Mistral’s new 1T model aims to leapfrog closed and open rivals French AI lab Mistral AI has released Mistral Large 4, a new large multimodal model aiming to leapfrog both American and Chinese rivals. 6 Vercel — AI dev-tools 8d ago Mistral Large 4 now available on AI Gateway Mistral Large 4 is now available on AI Gateway. The open-weight, natively multimodal model combines reasoning with text and image understanding. It is useful for cyber defense, manufacturing, and finance workflows. You can access the model through AI Gateway with one API key,… 8 llama.cpp releases dev-tools 9d ago v0.6.0 Overview llama.cpp v0.6.0 introduces the new llama_batch_ext extended batch API (with llama_process ) for mixed token/embedding inputs and MTP/deepstack state embeddings, adds support for the GLM-5.3-Flash (GLM5-Next) 320B hybrid model, the Clef decision model (text and vision)… 27 llama.cpp releases dev-tools 9d ago b11418: server: support vision input for Clef (#29969) server: support vision input for Clef move input_attn_causal to private extend old server_batch::embd server_batch::token::pos to multi dim nits fix abort fix img tokens cap fix yield_to_queue mutate data 25 r/LocalLLaMA community 9d ago Which model, which harness? I have data for you. I keep testing harness & model setups on real tasks: basic math, vision, computer use (reading emails, navigating stores) and coding. I created a benchmark and a leaderboard. You can see it here: https://airbench.ai/leaderboard?k=poL It measure capabilities (a % of sucess on the… 9 llama.cpp releases dev-tools 9d ago b11415: server: reject partial media truncation (#24076) server: reject partial media truncation server: keep only the keep_first fix Drop the mtmd test helper change, which no longer builds since clip_image_f32_batch stores its entries by value, and drop the vision test: no test fixture reaches a cut between two adjacent media chunks… 9 arXiv — Machine Learning research 9d ago Slow-Fast Multi-Teacher On-Policy Distillation for Capability Preservation arXiv:2610.02324v1 Announce Type: new Abstract: Foundation multimodal large language models are designed to support a broad spectrum of capabilities across diverse domains. Multi-teacher on-policy distillation (MOPD) provides an effective framework for consolidating… 15 arXiv — Machine Learning research 9d ago VisAudit: Evaluating Multimodal Agents for Visual Diagnosis and Repair arXiv:2610.02399v1 Announce Type: new Abstract: Multimodal agents are increasingly used for data visualization tasks but remain limited in autonomous review. Unlike humans, they may fail to recognize when a visualization is incorrect, determine what to change, repair it without… 15 arXiv — Machine Learning research 9d ago Structural-Functional Brain Connectivity Generation via Multimodal Hypergraph-based Flow Matching arXiv:2610.02722v1 Announce Type: new Abstract: Structural connectivity (SC) and functional connectivity (FC) provide complementary information on interactions between brain regions and are widely used in neuroimaging studies of neuropsychiatric disorders. Generative modelling… 13 arXiv — Machine Learning research 9d ago No-Free-Graph: Learning When Multimodal Data Should Be Graphified arXiv:2610.02768v1 Announce Type: new Abstract: Multimodal graph learning has recently emerged as an effective paradigm for in corporating inter-entity relationships into multimodal representations. Existing studies have made substantial progress on how to construct and optimize… 35 arXiv — NLP / Computation & Language research 9d ago Learning When to Commit from Partial Speech for End-to-End Simultaneous Speech Translation arXiv:2610.02612v1 Announce Type: new Abstract: Simultaneous speech translation must emit useful target text before the source is complete while preserving every committed token. We adapt a full-utterance speech language model using prefix supervision derived from its own… 15 arXiv — NLP / Computation & Language research 9d ago How Robust Is Multimodal Claim Verification to LLM Rewriting? arXiv:2610.02841v1 Announce Type: new Abstract: LLMs are known to introduce stylistic changes into generated text, yet how these stylistic shifts affect model decisions on scientific tasks remains underexplored. In this paper, we focus on multimodal claim verification, where the… 26 arXiv — NLP / Computation & Language research 9d ago Evaluating VQA in Vision Language Models using Cooperative Principles arXiv:2610.02878v1 Announce Type: new Abstract: We evaluate the performance of Vision Language Models in Visual Question Answering (VQA) when questions violate Grice's maxims. To do this, we use VLMs to generate question modifiers that add non-essential, ambiguous or false… 26 arXiv — NLP / Computation & Language research 9d ago Recursive Self-Improvement in Unified Multimodal Models arXiv:2610.03002v1 Announce Type: new Abstract: Unified multimodal models (UMMs) understand and generate both text and images, which lets a model produce its own training data. Existing self-improvement in UMMs keeps supervision on the visual side, where image understanding… 35 arXiv — NLP / Computation & Language research 9d ago CLIMB: Confidence-Guided Complementary Evidence for Multimodal Retrieval-Augmented Generation arXiv:2610.03421v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have shown strong visual reasoning abilities, but knowledge-intensive visual question answering often requires external textual evidence beyond the image and the model's parametric… 17 arXiv — NLP / Computation & Language research 9d ago Author Representation Strategies for Zero-Shot Authorship Attribution: A Comparative Study of LLM-Based and Embedding-Based Approaches arXiv:2610.03531v1 Announce Type: new Abstract: Authorship Attribution (AA) requires capturing fine-grained stylistic characteristics, making it particularly challenging in zero-shot (ZS) settings where no task-specific supervision is available. In this work, we investigate the… 27 r/LocalLLaMA community 10d ago For dual DGX spark users; GLM 5.3 flash got a 50%+ performance boost For the last few months, I ran DeepSeek v4.0 flash (NVFP4). First 0731 , then visionexp because it was a free improvement. I got around 65 tps decode and almost 2k prefill, and ran 4-5 agents in parallel, totalling around 200 tps cumulative decode. Because of this, I did not… 34 r/MachineLearning community 10d ago Here are some pictures of a robot costume wearing high-specularity edge-case mirror suit, a dataset (425 RAW/JPEGs) for benchmarking CV & depth-estimation algorithms against extreme mirror reflections [D] Purpose-built to stress-test computer vision models, depth cameras, and spatial AI against severe specular glare and geometric reflections. This 425-asset production archive features a custom faceted mirror suit captured in high-contrast outdoor environments to trigger… 23 arXiv — Machine Learning research 11d ago MOVE: Multimodal Open-world Verification and Expansion for Graph Learning arXiv:2610.00268v1 Announce Type: new Abstract: Multimodal graph learning faces a fundamental challenge: new classes may emerge after deployment, while models are trained with a fixed label space. Existing approaches typically detect unknown nodes and use LLMs to generate… 30 arXiv — Machine Learning research 11d ago Coupling Perception and Reasoning in Federated Multimodal Graph Foundation Models arXiv:2610.00277v1 Announce Type: new Abstract: Federated multimodal graph foundation models (GFMs) aim to adapt pretrained multimodal models to decentralized graph data, where each client owns a private multimodal graph and cannot share raw information. These models typically… 20 arXiv — Machine Learning research 11d ago OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One arXiv:2610.00381v1 Announce Type: new Abstract: Medical models are judged not only on correctness, but on whether reported confidence matches actual accuracy. Generalist multimodal medical models have expanded what a single model can perceive, yet they still express bounded… 31 arXiv — Machine Learning research 11d ago MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation arXiv:2610.00604v1 Announce Type: new Abstract: Vision-language-action policies often see only one or a few recent frames, which makes it difficult to evaluate how they use information that disappears during a task. We introduce MIKASA-Robo-VLA, a benchmark of 90… 36 arXiv — Machine Learning research 11d ago Beyond Unimodal Bases: Pullback Geometry for Multimodal Data arXiv:2610.00708v1 Announce Type: new Abstract: Data-driven Riemannian geometry provides nonlinear interpolation and geometric representations of high-dimensional data. For these operations to be statistically meaningful, paths between observations should preferentially traverse… 10 Page 2 of 10 · 500 articles ← Newer Older →
