News / #multimodal Tag Multimodal 500 articles archived under #multimodal · RSS Sign in to follow TechCrunch — AI news-outlet 21h ago Science Corporation implants its vision-restoring chip in its first Stargardt disease patient Max Hodak, the former president and co-founder of Neuralink, detailed on stage at TechCrunch Disrupt how his company's work with PRIMA is part of its path toward a novel brain-computer interface (BCI). 20 r/LocalLLaMA community 1d ago Basalt: Fast Blackwell inference engine for Qwen 3.8 Flash Next (and how we got vision for free) Dynamic vision, in action Basalt is an inference engine for 3.8 Flash Next, forked from Strata and heavily tuned for Blackwell. It supports vision, and for this to work you need to have the image encoder either in VRAM or CPU all the time, having it on VRAM takes valuable space… 6 r/LocalLLaMA community 2d ago Running unsloth/Qwen3.8-27B-UD-IQ4_XS 262k q4_0 KV + vision on 16GB VRAM @ 80t/s and 1500pp I get a lot of hate whenever I tell people I run a Q4 quant of Qwen3.8 27B with 262k KV 80t/s with vision, all while running 2 screens on windows on the same card. People seem to think I am lying or making it up. I am really just trying to help people. I am saddened whenever I… 31 r/LocalLLaMA community 2d ago Receipt scanner that runs a fine-tuned 0.8B vision model on the phone (free) - detailed cost tracking I wanted to track grocery spending per item without sending my receipts to anyone, so I built an Android app around a small vision model that reads a photo of a receipt into JSON: store, date, total, every line item. (iOS will take a couple more days to be checked by the Apple… 14 r/LocalLLaMA community 2d ago Decision 3.0: open, multimodal decision models from 0.6B to 27B. Ranks #1 on both Jev Decision Index boards With the vLLM Semantic Router team we released Decision 3.0 (d3) , our new SOTA decision model family ranging from 0.6B to 27B, Apache-2.0, all reading text, images and video. How it works - Built on Qwen3.5 (0.8B / 2B / 4B / 9B) and Qwen3.8-27B. The parameter counts include the… 38 arXiv — Machine Learning research 2d ago Reinforcement Learning via Fixed-point Flow Matching arXiv:2610.12848v1 Announce Type: new Abstract: In maximum entropy and behavior-constrained reinforcement learning (RL), the optimal policy is given by a Boltzmann distribution. Generating samples from this distribution can be challenging, as it is often complex and multimodal.… 32 arXiv — Machine Learning research 2d ago GIST: Generator-state-Informed Supervision for Tabular Foundation Models arXiv:2610.12904v1 Announce Type: new Abstract: Tabular foundation models (TFMs) based on prior-data fitted networks typically use sampled hard query labels, leaving avoidable sampling variability in pretraining. A key observation is that intermediate generator states can reveal… 27 arXiv — Machine Learning research 2d ago Evidential Multimodal Transformers with Dempster-Shafer Fusion for Emotion Recognition arXiv:2610.13013v1 Announce Type: new Abstract: Softmax-based multimodal emotion recognition methods often assign high confidence even when sensor data are corrupted or contradictory, for example when facial expressions suggest positive valence while vocal prosody indicates… 35 arXiv — Machine Learning research 2d ago Continually Post-training Small Models via Synthetic Self-distillation arXiv:2610.13113v1 Announce Type: new Abstract: Continual post-training faces a dual challenge of acquisition and retention: models must learn new capabilities while preserving prior knowledge. While Self-Distillation Fine-Tuning (SDFT) mitigates this using self-supervision with… 25 arXiv — Machine Learning research 2d ago Rethinking Relative Policy Differences Transfer of On-Policy Distillation arXiv:2610.13205v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a target policy on its own rollouts using token-level supervision from a stronger teacher, but typically relies on the teacher being substantially more capable than the target. Recent work… 37 arXiv — Machine Learning research 2d ago Monarch-MoE: Structured Experts for Multimodal Continual Instruction Tuning arXiv:2610.13215v1 Announce Type: new Abstract: Learning new tasks in multimodal continual instruction tuning can cause catastrophic forgetting of previously acquired knowledge. Although multi-expert LoRA distributes learning across experts, updates to a dense low-rank expert… 38 arXiv — Machine Learning research 2d ago MHR-Router: Multimodal Heterogeneous Retrieval-Enhanced Adaptive Routing for VLM Inference arXiv:2610.13278v1 Announce Type: new Abstract: Vision-language models (VLMs) exhibit complementary capabilities, yet selecting the most suitable expert for a given image-question query remains challenging due to heterogeneous multimodal inputs and varying expert strengths.… 16 arXiv — NLP / Computation & Language research 2d ago Unsupervised Multimodal Reward Modeling arXiv:2610.12707v1 Announce Type: new Abstract: Vision-language reward models are trained on annotated preference pairs, which are expensive to collect, can disagree, and are ranked by benchmark accuracy. We build one with no human preference label: a linear head on an… 22 arXiv — NLP / Computation & Language research 2d ago Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction arXiv:2610.13672v1 Announce Type: new Abstract: Large language models struggle to reliably identify and correct errors in their own reasoning. Unlike prior training-based approaches that fine-tune an LLM to generate revisions or feedback, we learn where to intervene in failed… 19 arXiv — NLP / Computation & Language research 2d ago When VLMs Destroy What They Know: Spatial Representations Fall Below Lexical Priors arXiv:2610.13910v1 Announce Type: new Abstract: Vision-Language Models (VLMs) increasingly analyze and reason about images, yet it remains unclear whether their internal representations faithfully encode the structured relations required for such reasoning. We introduce EICS… 21 arXiv — NLP / Computation & Language research 2d ago MVGraph: Benchmarking LVLMs on Evidence Composition across Multi-View Visual Graph arXiv:2610.12961v1 Announce Type: cross Abstract: We introduce MVGraph, a benchmark for evaluating large vision-language models (LVLMs) on multi-graph reasoning where the evidence required by a query is distributed across multiple interrelated visual graph views. Although such… 31 arXiv — NLP / Computation & Language research 2d ago GeoGR: Spatial-Semantic Token Compression and Group Routing for Efficient 3D VLMs arXiv:2610.13035v1 Announce Type: cross Abstract: Multi-view representations enable pretrained image and video vision-language models to understand 3D scenes, but their long visual token sequences make inference costly. Redundancy arises both from repeated observations of the… 6 arXiv — NLP / Computation & Language research 2d ago Locate, Judge, Read: Role-Specialized Tool Agents over a Knowledge-Point Index for Open-Domain Multimodal Document QA arXiv:2610.13128v1 Announce Type: cross Abstract: Question answering over large heterogeneous document collections requires first finding the right passages and then reading them, yet current retrieval-augmented agents give one model both kinds of tools and let it schedule… 38 arXiv — NLP / Computation & Language research 2d ago Vision Transformers Learn Word Meaning from Images Alone arXiv:2610.13252v1 Announce Type: cross Abstract: Can a vision model learn what words mean from pixels alone? We show that the two largest DINOv3 vision transformers organize words rendered in images by word meaning, despite being pretrained using self-supervision on images… 5 The Information — AI news-outlet 2d ago Amazon’s Bargains Include Its Stock It’s the quiet before the storm of earnings season, which starts in a week or two. While we wait for the third-quarter updates, here’s a thought: Investor enthusiasm for semiconductor stocks and blue-sky vision companies like SpaceX appears to have sucked money out of stocks of… 24 The Information — AI news-outlet 2d ago Amazon’s Bargains Include Its Stock It’s the quiet before the storm of earnings season, which starts in a week or two. While we wait for the third-quarter updates, here’s a thought: Investor enthusiasm for semiconductor stocks and blue-sky vision companies like SpaceX appears to have sucked money out of stocks of… 13 r/LocalLLaMA community 4d ago Fully local conversational AI: Whisper + Hermes 8B + Kokoro, zero cloud, running inside a plush toy A conversational AI plush toy where nothing leaves my network. Everything runs on my LAN: STT: Whisper LLM: Hermes 3 Llama 3.1 8B (Q8_0) TTS: Kokoro Memory: per-person (LangGraph). It knows who it's talking to and remembers past conversations Vision: face identity (dlib) +… 18 r/LocalLLaMA community 4d ago Qwen 3.6 35B A3B: 131K context + vision on 6GB VRAM Running HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive Q4_K_M with llama.cpp at ~600 tok/s prefill and 23 tok/s decode, 131k context window, Q8 KV cache - on an RTX 2060 6GB + 32GB DDR4 RAM. Speeds start at ~600 tok/s prefill / 23 tok/s decode on an empty KV cache. As… 21 r/LocalLLaMA community 5d ago Just another purely open-weight models benchmark Live results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule areas): https://beta.locallm.top Evaluated by domain experts (my friends mostly;… 14 arXiv — Machine Learning research 5d ago Leakage-Controlled Multimodal Learning for Diagnosis and Progression Prediction in Alzheimer's Disease Research arXiv:2610.10648v1 Announce Type: new Abstract: Alzheimer's disease prediction involves irregular visits, heterogeneous measurements and incomplete modalities. This study presents a multimodal multitask framework combining an adapted SFCN MRI encoder, four causal clinical… 27 arXiv — Machine Learning research 5d ago PXtal: Learning to Align Powder X-Ray Diffraction and Crystal Structures under Information Asymmetry across Modalities arXiv:2610.10653v1 Announce Type: new Abstract: Scientific multimodal learning commonly assumes that paired views are comparably informative. Powder X-ray diffraction (PXRD) makes this mismatch explicit: compressing a three-dimensional crystal structure into a one-dimensional… 25 arXiv — Machine Learning research 5d ago Explaining the Saliency Map Sparsity of Adversarially-Trained Neural Networks arXiv:2610.10666v1 Announce Type: new Abstract: Understanding why deep neural networks make a given prediction is of great importance for their safe deployment. In computer vision, saliency maps, which highlight the image region most influential for a prediction, remain a… 7 arXiv — NLP / Computation & Language research 5d ago ActiveMedAgent: Cost-Aware Trajectory Learning for Multimodal Medical Diagnosis arXiv:2610.11140v1 Announce Type: cross Abstract: Clinical diagnosis is inherently sequential: clinicians escalate from cheap to costly tests only when additional evidence is expected to resolve diagnostic uncertainty. We present ActiveMedAgent, a framework that brings this… 15 arXiv — Machine Learning research 5d ago PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation arXiv:2610.11167v1 Announce Type: new Abstract: Vertical-domain few-shot classification remains challenging for small language models, as limited supervision makes it difficult to acquire domain-specific decision knowledge. On-Policy Distillation (OPD) can improve teacher-guided… 11 arXiv — Machine Learning research 5d ago Multimodal Graph Retrieval-Augmented Sequential Recommendation via Collaborative Filtering Paths arXiv:2610.11228v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong potential for sequential recommendation through their ability to reason over complex multimodal data. However, existing approaches either rely solely on the target… 19 arXiv — Machine Learning research 5d ago Deflating the Hessian: Rank-4 W4A4 Quantization for Multimodal Diffusion Transformers arXiv:2610.11315v1 Announce Type: new Abstract: In diffusion transformers, low-rank branches can mitigate 4-bit weight--activation (W4A4) post-training quantization (PTQ) loss by decomposing each weight into a low-bit residual and a high-precision low-rank component. Existing… 26 arXiv — Machine Learning research 5d ago SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces arXiv:2610.11366v1 Announce Type: new Abstract: Spatial coding agents significantly improve spatial reasoning in Multimodal Large Language Models (MLLMs) by using external tools to generate verified execution traces. However, this paradigm inherently suffers from prohibitive… 31 arXiv — Machine Learning research 5d ago CoPoE: Multimodal Fusion via Decomposable Disease-Coordinate Product-of-Experts for Missing-Modality Alzheimer's Diagnosis arXiv:2610.11394v1 Announce Type: new Abstract: Multimodal Alzheimer's disease (AD) diagnosis benefits from integrating heterogeneous clinical, imaging, genomic, and biomarker evidence, but clinical cohorts frequently suffer from irregular modality missingness. Existing fusion… 34 arXiv — NLP / Computation & Language research 5d ago Disentangling Linguistic and Paralinguistic Information with Routed Sparse Autoencoders arXiv:2610.10865v1 Announce Type: new Abstract: Self-supervised speech encoders contain linguistic and paralinguistic information in a shared, entangled representation space. We combine a TopK sparse autoencoder with route-specific supervision and cross-factor adversaries.… 28 arXiv — NLP / Computation & Language research 5d ago Stochastic Teacher Intervention for Agentic On-Policy Distillation arXiv:2610.10878v1 Announce Type: new Abstract: On-policy distillation (OPD) efficiently transfers capabilities from a stronger teacher to a student language model through dense token-level supervision on student-generated rollouts and has shown promise on complex tasks such as… 23 arXiv — NLP / Computation & Language research 5d ago BeliefScope: Diagnosing Evidence-Driven Revision and Pressure-Induced Shifts in Large Language Models arXiv:2610.11305v1 Announce Type: new Abstract: A language model may revise the same proposition after receiving genuinely relevant evidence or after receiving directional user pressure that adds no relevant fact. The observable response shift alone therefore does not reveal… 28 arXiv — NLP / Computation & Language research 5d ago MetaEncoder: Exploring the Limit of Bi-Encoders for Multimodal System One Decision Making with Natural Language Interface arXiv:2610.11316v1 Announce Type: new Abstract: System One models output constrained decisions and probability distributions rather than free-form text generation. While prevailing paradigms rely on structured schema objects to encode state, intent, and candidate choices, we… 5 arXiv — NLP / Computation & Language research 5d ago AdaptEvo: Adaptive Agent Learning with Evolving Supervision arXiv:2610.11354v1 Announce Type: new Abstract: Rule-governed contextual decision tasks require models to apply specified rules to case-specific context and evidence. Written rules can leave gaps in decision guidance and process evaluation, while reference judgments vary in… 11 arXiv — NLP / Computation & Language research 5d ago From Sparse Representations to Behavioral Insights for Multimodal Depression Assessment arXiv:2610.11787v1 Announce Type: new Abstract: Multimodal depression assessment offers a promising approach to analyzing behavioral patterns associated with depression. However, existing methods often rely on dense and opaque multimodal representations, making it difficult to… 26 arXiv — NLP / Computation & Language research 5d ago When History Helps and Hurts: Selective History Use across Multimodal Turns arXiv:2610.11948v1 Announce Type: new Abstract: Reliable multimodal interaction depends on selective use of conversational history: an earlier question may remain relevant while its previous answer is outdated, whereas a current request may depend on historical evidence despite… 16 arXiv — NLP / Computation & Language research 5d ago EgoVoice: Proactive Spoken Assistance from Egocentric Multimodal Streams arXiv:2610.12248v1 Announce Type: new Abstract: Wearable augmented reality (AR) assistants are moving toward continuous real-world interaction, where they perceive the user's activity through first-person video and audio and provide timely spoken guidance without being… 20 arXiv — NLP / Computation & Language research 5d ago WorldBench: Evaluating LLMs on Three.js Voxel World Generation arXiv:2610.10622v1 Announce Type: cross Abstract: Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges take one view of the output: a vision-language model scores a few rendered… 35 arXiv — NLP / Computation & Language research 5d ago GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary arXiv:2610.11129v1 Announce Type: cross Abstract: Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge. Existing AI-Generated Game Commentary (AI-GGC) studies remain fragmented across games, modalities,… 29 Vercel — AI dev-tools 5d ago Liquid AI's d1 is available on AI Gateway Liquid d1 is now available on AI Gateway . d1 is a decision model that evaluates shared state against typed questions for classification, routing, and scoring. It returns structured answers with probabilities without generating text tokens. d1 also has vision support: the model… 31 r/MachineLearning community 5d ago The Alchemy of Semi-Supervision [R] Hey r/MachineLearning , I wrote this blog post on semi-supervised learning. Blog post: https://stefankeselj.com/entries/semi_supervision . https://preview.redd.it/2fh9jw7ahbuh1.png?width=1502&format=png&auto=webp&s=529115c47d0029c69e84803ec93ae81949f92bbf I think it's a useful… 19 arXiv — Machine Learning research 6d ago Algorithmic Scratchpads and Curriculum Staging for Arithmetic Reasoning in Tiny Transformers arXiv:2610.09003v1 Announce Type: new Abstract: Autoregressive Large Language Models (LLMs) frequently struggle with deterministic multi-step algorithmic tasks such as multi-digit multiplication and long division. In this paper, we investigate the mechanics of multi-step… 7 arXiv — Machine Learning research 6d ago Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding arXiv:2610.09355v1 Announce Type: new Abstract: Learning EEG representations that generalize across cognitive tasks, subjects, and recording conditions remains a key challenge in electroencephalography (EEG) decoding. Recent advances in foundation models have improved EEG… 8 arXiv — Machine Learning research 6d ago Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Data arXiv:2610.09411v1 Announce Type: new Abstract: Multimodal representations enable zero-shot classification and retrieval, but aligning independently trained models usually requires large amounts of paired data. Yet, the Platonic Representation Hypothesis suggests that models… 32 arXiv — Machine Learning research 6d ago DSReg: Provably Recovering Individual World Latents without Reconstruction arXiv:2610.09457v1 Announce Type: new Abstract: Methods that recover individual latent variables of the world, from nonlinear ICA to dictionary learning and causal representation learning, anchor the latents to observations through reconstruction, auxiliary supervision, or… 32 arXiv — Machine Learning research 6d ago Align Before You Combine: Reference Space Calibration for Supervision Without Ground Truth arXiv:2610.09525v1 Announce Type: new Abstract: We introduce a calibration-first framework that produces supervision scores without access to ground-truth labels or a shared annotation space. Our framework aligns subset-specific scorers using a synthetic ordinal reference space… 19 Page 1 of 10 · 500 articles Older →
