News / #long-context Tag Long Context 500 articles archived under #long-context · RSS Sign in to follow Vercel — AI dev-tools 1d ago Gemini 4 Argon now available on AI Gateway Gemini 4 Argon from Google is now available on AI Gateway . Gemini 4 Argon is built for coding, agentic work, and complex reasoning. It has a 1M token context window, accepts text, image, PDF, and video input, and supports tool calling. On coding tasks, Gemini 4 Argon builds… 4 r/LocalLLaMA community 2d ago strix-llama new branch: prompt processing +21–54% over Vulkan and up to 2.7× over stock ROCm kernel at 128k, same outputs quality I've been tuning llama.cpp on my Strix Halo box (Radeon 8060S, 128 GB) and the numbers are good enough to share. It's a branch called rdna35-perf on top of halo-box/strix-llama.cpp , a llama.cpp fork aimed at Strix Halo: https://github.com/asf0/strix-llama.cpp/tree/rdna35-perf… 28 arXiv — Machine Learning research 2d ago Double-Take: Improving Capacity, Binding, and Selectivity In Long-Context Within Bidirectional Memory arXiv:2610.13073v1 Announce Type: new Abstract: Processing a long sequence does not guarantee that distant information remains usable. As context grows, a fixed-size memory must retain many associations, connect information across positions, and preserve relevant structure… 33 arXiv — NLP / Computation & Language research 2d ago S$^3$R: Fast and Faithful RL Rollouts via Self-Sparse Speculative Decoding arXiv:2610.13202v1 Announce Type: cross Abstract: Fast rollout generation for long-context reinforcement learning with verifiable rewards (RLVR) requires reducing KV-cache traffic without changing the response groups used for policy optimization. In our profiled RLVR workloads,… 9 arXiv — NLP / Computation & Language research 2d ago Taking Tokenization off the Host for Agentic LLM Serving arXiv:2610.13325v1 Announce Type: cross Abstract: Agentic workloads resend long contexts that barely change between calls. Prefix caching now skips most of their prefill, yet every request must still be tokenized in full before the cache can even be consulted. Tokenization, a… 5 Vercel — AI dev-tools 2d ago Muse Wonder now available on AI Gateway Muse Wonder from Meta is now available on AI Gateway. The model is designed for coding, long-running agents, and computer use. It can plan multi-step tasks, delegate work to sub-agents, and recover from errors as work progresses. Muse Wonder has a 1M-token context window and… 5 r/LocalLLaMA community 3d ago Qwen 3.8 27B Q5 vs Qwen 3.8 Next Q3_S for document analysis I have an app that processes ~500 different documents every day. An agent analyzes them, sorts them, tags them, extracts various legal nuances, creates descriptions, etc. So far I've been using Qwen 3.8 27B Q5 with a 150K context window, but I'm wondering whether I should switch… 17 r/MachineLearning community 4d ago I Built a O(NlogN) attention system that retains 97% accuracy over long context (MQAR)[P] ALHR- Adaptive learnable Hierarchical Routing is a static binary tree based system that uses learnable functions to REDUCE the amount of keys used. It takes less memory and SCALES much better VRAM with tokens   submitted by   /u/Alarming-Emotion-894 [link]  … 12 r/LocalLLaMA community 4d ago Is anyone running Qwen3.8 Flash Next with a 1M context? So, while doing a search to read the model card again, and because I didn't memorize the huggingface URL, I saw an AI "answer" at the top of the search, stating that while it natively supports a ~244k context, it could go to 1M using YaRN. That's the first I have heared of both,… 5 r/LocalLLaMA community 4d ago Qwen 3.6 35B A3B: 131K context + vision on 6GB VRAM Running HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive Q4_K_M with llama.cpp at ~600 tok/s prefill and 23 tok/s decode, 131k context window, Q8 KV cache - on an RTX 2060 6GB + 32GB DDR4 RAM. Speeds start at ~600 tok/s prefill / 23 tok/s decode on an empty KV cache. As… 21 r/LocalLLaMA community 4d ago Qwen3.8 27b with 200K ctx + MTP on 12GB Ampere Cards Hello all! Thank you to everyone that contributed their feedback and notes for LlamAmpere the last time I posted. I've continued to chip away at improvements for the 3090 crowd -- this release is modestly faster (3-4%), with a few hundred MB smaller runtime (if you're using… 5 r/LocalLLaMA community 5d ago LlamaTale v0.43.0 - MCP Server for story creation In the dawn of the LLM craze, when wild Llama2's roamed free and 4k tokens was considered a fairly long context for local models, I forked an interactive fiction and MUD library called Tale to experiment with LLM generated content. The idea being a text based adventure with… 23 arXiv — Machine Learning research 5d ago Learning infinite context windows in recurrent architectures via spatial neural computing arXiv:2610.10690v1 Announce Type: new Abstract: Recurrent neural networks (RNNs) offer linear-time scaling with sequence length while requiring only constant memory, yet they struggle to capture long-range dependencies due to vanishing gradients and limited receptive fields. To… 35 arXiv — Machine Learning research 5d ago QUILT: Rethinking Sparse-Attention Prefill through Shared Query Execution arXiv:2610.11134v1 Announce Type: new Abstract: Sparse attention reduces the cost of long-context attention, but existing kernels typically process queries independently, repeatedly loading and dequantizing KV entries shared across queries. We observe substantial overlap in the… 18 arXiv — NLP / Computation & Language research 5d ago Real Long-Term Memory for AI: A 50-Million-Token Window That Is Faster and Cheaper Than Recompute arXiv:2610.10845v1 Announce Type: new Abstract: A large language model can only use the text that fits in its context window, and it recomputes its internal key-value (KV) state for a prompt every time the prompt is sent. We test a memory layer, the public package galahad-kv,… 8 arXiv — NLP / Computation & Language research 5d ago REMORY: Learning Residual Memory for Context Compaction arXiv:2610.11287v1 Announce Type: new Abstract: Long-horizon agents compact their history to continue within a finite context window, but a textual summary alone may not support every subsequent decision. We introduce REMORY, a neural memory network that supplements the summary… 32 arXiv — NLP / Computation & Language research 5d ago BioBigBird: A Sparse Attention Model for Long-Range Dependency Processing in Biomedical Text arXiv:2610.11430v1 Announce Type: new Abstract: While domain-specific Large Language Models (LLMs) have encoded vast biomedical knowledge, their limited context windows often hinder a deep understanding of nuanced relationships within and across texts. To address this… 25 arXiv — NLP / Computation & Language research 5d ago Probing for Long-Horizon Deductive Reasoning Capabilities in Language Models with Prolog arXiv:2610.11592v1 Announce Type: new Abstract: Current frontier LLMs can theoretically process long contexts with 1M tokens or more. But to what extent can they go beyond simple retrieval and perform deeper reasoning over such long contexts? We empirically investigate… 27 arXiv — NLP / Computation & Language research 5d ago VFold: Symmetry-Aware Cross-Layer Value Cache Compression arXiv:2610.12338v1 Announce Type: new Abstract: While caching key-value (KV) states accelerates Large Language Model (LLM) decoding, this cache can dominate memory usage at long context lengths. One solution is to compress this memory by exploiting inter-layer cache… 5 arXiv — Machine Learning research 6d ago KVFetch: Temporal Prefetching for the Missing Half of KV Cache Compression arXiv:2610.08811v1 Announce Type: new Abstract: As context windows scale to tens or hundreds of thousands of tokens, KV cache compression has become essential for efficient LLM inference. Existing methods fall into three families: score-based eviction, summary compensation, and… 25 arXiv — NLP / Computation & Language research 6d ago Decoupling Logic from Persona: Structural Immunity of Edge LLM Agents to Context Pollution arXiv:2610.09772v1 Announce Type: new Abstract: Small language-model agents on edge devices must hold a persona and reason correctly at once, inside one context window that fills with conversational history and persona instructions. We study what happens to the logical part of… 10 arXiv — NLP / Computation & Language research 6d ago Mechanics of Long-Context Hybrid Models Part 1.1: From Hybrid Attention to Hybrid Position arXiv:2610.10114v1 Announce Type: new Abstract: The architectural design of Large Language Models (LLMs) is shifting from traditional full-attention-only models to hybrid models, which combine different attention modules to improve long-context efficiency and performance in… 18 arXiv — Machine Learning research 7d ago AttSVD:Prompt-Adaptive Low-Rank KV Cache Compression via Attention-Guided SVD arXiv:2610.06927v1 Announce Type: new Abstract: The key-value (KV) cache of autoregressive transformers grows linearly with context length and dominates memory at long context. Most training-free remedies evict low-importance tokens, an irreversible choice along the sequence… 6 arXiv — NLP / Computation & Language research 7d ago ReFold: Training-Free Reversible Inter-Turn Context Folding for Long-Horizon Agents arXiv:2610.07863v1 Announce Type: new Abstract: Long-horizon LLM agents act on an append-only interaction history that is re-sent to the model at every step, so the context and its cost grow with steps until the sessions exceed the context window. Existing methods manage the… 8 arXiv — NLP / Computation & Language research 7d ago UNREAL: Unifying Retrieval and Long-Context with a Single Model arXiv:2610.08463v1 Announce Type: new Abstract: Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an entire corpus. We ask whether a single model-internal mechanism can select… 19 Vercel — AI dev-tools 7d ago Glyph Cluster is now available in stealth for free on AI Gateway Glyph Cluster is now available on AI Gateway as a stealth model for teams on Pro and Enterprise plans with purchased AI Gateway credits. It is free to use during the stealth period. Glyph Cluster is a reasoning model for coding and long-context analysis, and handles multi-step… 23 r/LocalLLaMA community 8d ago Aplomb 1: open-weights 5.3B decision model, 1M context, text/image/video/audio in one request, #1 among 4B models on the Decision Index We released Aplomb 1 today, a 5.3B decision model with open weights. It reads up to 1M tokens of text, JSON, images, video and audio in a single request, and on our API it makes a decision on a 1M-token document in about 3 seconds, at $0.02 per 1M input tokens with free output… 36 r/LocalLLaMA community 8d ago Qwen3.8-Flash-Next on Strata Hey! 👋 I have released an official support for Strix Halo machines on Strata for Qwen3.8-Flash-Next. Currently numbers are the best on long context decode and ppts using typical Unsloth’s Q4 and GSQ-RCO model weights. Can go up to 1M context length without big speed loss.… 32 r/LocalLLaMA community 8d ago What happens when a LLM watches its own context window run out? I made Terminal Soliloquy , a terminal artwork that connects to llama.cpp and displays a model's monologue as its context window fills. It has a retro phosphor look, and runs in a terminal window. I'm actually running it full-screen on a Raspberry Pi display inside an old 1960s… 6 r/LocalLLaMA community 9d ago Switched my local agent from Qwen3.8 27B to Ornith 1.5 35B-A3B on two 5070 Tis: about 180 tok/s vs 60, same scores on my tests My setup is two RTX 5070 Ti 16GB cards (the second one is on an OCuLink dock) with 64GB of RAM, Ollama on Windows, and the agent runs on pi in WSL. Until last night the daily model was Qwen3.8 27B UD-Q4_K_XL at 128K with MTP, which does about 55 to 70 tok/s across both cards. I… 24 arXiv — NLP / Computation & Language research 9d ago Trained Agentic Context Management arXiv:2610.02404v1 Announce Type: cross Abstract: We study long context language models. Instead of training long context natively, or designing a long context harness, we train a model over the simplest possible harness: a tool to call itself with any specified prompt and a… 7 arXiv — NLP / Computation & Language research 9d ago Tailoring the Quantization Space for 1-Bit KV Cache Compression arXiv:2610.03027v1 Announce Type: cross Abstract: The key-value (KV) cache becomes a major memory bottleneck in long-context LLM inference, placing substantial pressure on memory capacity and bandwidth. To mitigate this bottleneck, vector quantization (VQ) has emerged as a… 20 r/LocalLLaMA community 11d ago Aleph-Alpha/Kolibri-1 · Hugging Face - 78B parameters. 3.46B active. Up to 1M tokens of context - Apache 2.0 From Aleph Alpha on 𝕏: https://x.com/Aleph__Alpha/status/2106306840657297814 Tech report: https://aleph-alpha.com/downloads/tech-report.pdf   submitted by   /u/Nunki08 [link]   [comments] 15 r/LocalLLaMA community 11d ago Fixed long-horizon task drift on local setups using a deterministic state plugin Ran into an annoying issue with local models on long tasks. Once context window compaction hits after a few thousand tokens, the model loses sight of the original scope. Even with good system prompts, a few compaction cycles cause goal drift, hallucinated task completion, or… 10 arXiv — Machine Learning research 11d ago EchoPress: Query-Agnostic KV Cache Pruning via Virtual Context Reconstruction arXiv:2610.00412v1 Announce Type: new Abstract: KV cache pruning reduces long-context inference memory usage by evicting less important key-value pairs. KVzip estimates importance through context reconstruction: prompting a model to repeat the context chunk by chunk. This… 27 arXiv — Machine Learning research 11d ago CommunityKV: Efficient Long-Context Decoding via Graph Partitioning arXiv:2610.00418v1 Announce Type: new Abstract: Scaling Transformers to long contexts is constrained by the quadratic cost of self-attention and the linear growth of key-value cache memory transfer. Sparse attention mitigates this by retrieving only relevant tokens, but current… 22 r/LocalLLaMA community 12d ago Strata on a power limited 5090 and 96GB of DDR5-6400 is cranking out 150-200 tok/s decode and 5-6k prefill! Qwen3.8-Flash-Next at IQ3_S, CTX at 128k tokens (8-bit).   submitted by   /u/z0_o6 [link]   [comments] 10 arXiv — Machine Learning research 12d ago Hermes: Learning Contextual Reasoning Unlocks Test-Time Scaling arXiv:2609.38332v1 Announce Type: new Abstract: Test-time scaling improves model performance by allocating additional compute during inference. Using this compute effectively across multiple context windows requires deciding how to allocate fresh contexts and what information to… 8 arXiv — Machine Learning research 12d ago SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUs arXiv:2609.38830v1 Announce Type: new Abstract: Sparse attention is widely used to accelerate long-context inference in modern large language models (LLMs), but its input-dependent execution behavior introduces previously unexplored privacy risks. We identify a new GPU… 33 arXiv — NLP / Computation & Language research 13d ago Scaling Parameter and Context in Attention: Native Sparse Attention from Mixture-of-Head arXiv:2609.38832v1 Announce Type: new Abstract: Scaling attention parameters can improve language model quality, but retaining full token histories makes additional heads costly at long contexts. Furthermore, since attention retrieves and combines contextual information,… 14 arXiv — NLP / Computation & Language research 13d ago The Invisible Language Tax: Token Premiums of French and Regional Languages in 2026 LLM Tokenizers, and a French-Optimized Prototype arXiv:2609.39001v1 Announce Type: new Abstract: LLM services are billed per token and context windows are measured in tokens, yet the number of tokens needed for the same content varies across languages. We measure this token premium on seven tokenizers of widely used 2026… 36 arXiv — NLP / Computation & Language research 13d ago LatentHarness: Learning Latent Actions for Memory and Reasoning via Counterfactual Policy Distillation arXiv:2609.39740v1 Announce Type: new Abstract: Long-context reasoning faces two complementary bottlenecks: retaining evidence across long inputs and sustaining computation across many reasoning steps. Existing approaches largely address them separately, with external memory… 12 llama.cpp releases dev-tools 14d ago b11279 add GLM-5.3-Flash (GLM5-Next) support ( #27773 ) Rebase GLM-Next support onto master, and migrate to llama-memory-hybrid-idx Add initial MTP support Merge branch optimizations. Reduce allocated compute buffer size, speed up long context decode, fla, and slight MTP improvements.… 6 arXiv — Machine Learning research 14d ago Fractional State Space Transition for Long Sequence Modeling arXiv:2609.36314v1 Announce Type: new Abstract: State Space Models (SSMs) compress sequence history into a bounded recurrent state, making the resulting memory law a central architectural choice for long-context performance. Most modern SSMs rely on ODE-based dynamics that lead… 11 arXiv — Machine Learning research 14d ago Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression arXiv:2609.36322v1 Announce Type: new Abstract: Chunked KV-cache compression reduces the memory and attention costs of long-context inference by compressing windows of consecutive tokens into fewer cache entries at a fixed stride. Such compression also introduces a new… 18 arXiv — Machine Learning research 14d ago Triadic Linear Attention: Three-Dimensional Recurrent States for Long-Context Sequence Modeling arXiv:2609.36529v1 Announce Type: new Abstract: Recurrent neural networks (RNNs) compress the historical context into a memory state of fixed size, thus allowing for constant-time inference. The memory state size is a crucial factor in their performance, as exemplified by the… 23 arXiv — NLP / Computation & Language research 14d ago Beyond the Context Window: An Adaptive Entropy-Based Routing Framework for Hybrid Retrieval and Long-Context Language Models arXiv:2609.35831v1 Announce Type: new Abstract: Modern large language models now support context windows of more than one million tokens, which has raised the question of whether retrieval-augmented generation (RAG) is still necessary. Pure long-context (LC) processing is… 9 arXiv — NLP / Computation & Language research 14d ago Eternal Sunshine of the Spotless Mind: Systematically Erasing LLM's Memories arXiv:2609.36414v1 Announce Type: new Abstract: We consider persistent LLMs that accumulate memories of their interactions with a user over time. Such LLMs maintain memories using external storage, which they can query to overcome the limitations of a fixed context window. Such… 20 arXiv — NLP / Computation & Language research 14d ago MemFold: Learning Compact Soft Memory for Long-Context Personalization via On-Policy Optimization arXiv:2609.36435v1 Announce Type: new Abstract: An assistant that serves the same user over a long horizon has to answer from what that user has revealed: which preferences still hold, which were revised, and which constraints apply now. Retaining that information is not the… 34 arXiv — NLP / Computation & Language research 14d ago MultiTalk: Scaling Full-Duplex Speech Models to Long, Multi-Party, Bilingual Conversation arXiv:2609.36903v1 Announce Type: new Abstract: End-to-end full-duplex speech models have brought open-source machine conversation closer to human-like interaction, yet existing systems remain limited in two intertwined dimensions: long-context robustness and multi-party… 38 Page 1 of 10 · 500 articles Older →
