News / #long-context Tag Long Context 500 articles archived under #long-context · RSS Sign in to follow arXiv — NLP / Computation & Language research 3mo ago Uncertainty-gated selection for block-sparse attention arXiv:2607.07724v1 Announce Type: cross Abstract: Block-sparse attention scales long-context language models by replacing the O(N^2) softmax with a per-query top-k selection over key blocks. This cutoff is myopic: when the k-th and (k+1)-th blocks are nearly tied in score, the… 7 arXiv — Machine Learning research 3mo ago Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE arXiv:2607.07740v1 Announce Type: new Abstract: Modern LLMs are increasingly deployed in long-context applications such as retrieval-augmented generation, repository-level coding, and agentic workflows whose accumulated reasoning and tool traces routinely push the input an order… 37 arXiv — Machine Learning research 3mo ago Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing arXiv:2607.07953v1 Announce Type: new Abstract: Self-attention lets each token retrieve information from the full context, but its quadratic cost in sequence length limits training and inference at long context. This paper presents a comparative study of softmax attention and… 10 r/LocalLLaMA community 3mo ago Locally run assistant on a w-10 board on a local Xiaozhi server Hello, Locallama! "long-time" member here, from the days when the max context window doubled from 2K to a whooping 4K! Now I feel like I am living in a dream world, doing things I could not have imagined back then. This is Belochka, or Bella, running on a Mac and projected to an… 31 r/LocalLLaMA community 3mo ago Stripping terminal noise from agent context via a lazy-loaded local CLI layer. Looking for brutal feedback on this heuristic. When building long-running coding agents, terminal output is one of the fastest ways to poison a context window. If an agent runs an intensive build, an install command, or a massive search ( grep / find ), it easily generates hundreds of lines of raw log noise. The agent reads… 7 Hugging Face Daily Papers research 3mo ago Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity Abstract Sparse Delta Memory extends gated linear RNNs with sparse addressing to dramatically increase hidden state capacity for improved long-context learning and retrieval while maintaining computational efficiency. Generated by Qwen/Qwen2.5-Coder-32B-Instruct Linear attention… 30 arXiv — Machine Learning research 3mo ago Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference arXiv:2607.07144v1 Announce Type: new Abstract: The key-value (KV) cache dominates the memory cost of long-context autoregressive inference, and a growing body of work compresses it through quantization, eviction, or offloading. We study a complementary question: once a… 36 arXiv — Machine Learning research 3mo ago Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity arXiv:2607.07386v1 Announce Type: new Abstract: Linear attention models allow a fixed state size and a fixed amount of compute per token. However, due to their limited state size, linear attention models fall behind in long-context recall compared to softmax-attention-based… 5 arXiv — NLP / Computation & Language research 3mo ago Nectar: Neural Estimation of Cached-Token Attention via Regression arXiv:2605.09778v2 Announce Type: replace-cross Abstract: Evaluating softmax attention over a fixed long context requires reading every cached key-value pair for each new query token. For a given context (a book, a manual, a legal corpus) the attention output is a deterministic… 18 Vercel — AI dev-tools 3mo ago Muse Spark 1.1 is now available on AI Gateway Muse Spark 1.1 from Meta is now available on AI Gateway . It is a multimodal reasoning model with a 1M token context window built for agentic tasks, accepting text, image, video, PDF, and audio inputs. Muse Spark 1.1 plans and orchestrates work across tools and services,… 34 arXiv — Machine Learning research 3mo ago PatchOptic for Shared-State LLM Workflows with Projected Views and Verified Structured Updates arXiv:2607.05483v1 Announce Type: new Abstract: Agentic workflows often operate over shared, structured state. Because LLM context windows are limited, each model invocation is typically shown only the state fragment needed for the current workflow step, a pattern commonly known… 7 arXiv — NLP / Computation & Language research 3mo ago Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving arXiv:2607.05399v1 Announce Type: new Abstract: Large language model serving is increasingly limited by KV-cache growth under long-context workloads, yet existing KV-cache compression techniques are difficult to compare because they were evaluated on different models, tasks,… 30 arXiv — NLP / Computation & Language research 3mo ago ResonatorLM: Causal Resonant Field Mixing for Efficient Long-Context Language Modelin arXiv:2607.05583v1 Announce Type: new Abstract: Contemporary language models are dominated by the transformer architecture, which leverages self-attention mechanisms to enable more efficient, parallelized training across a wide set of documents and corpora. This has allowed… 20 arXiv — NLP / Computation & Language research 3mo ago Inject or Navigate? Token-Efficient Retrieval for LLM Analysis of Transactional Legal Documents arXiv:2607.05764v1 Announce Type: new Abstract: Answering questions over a set of transactional legal documents is most simply done by injecting the whole corpus into the LLM's context window on every query. That baseline maximises retrieval recall, but its token footprint… 31 arXiv — NLP / Computation & Language research 3mo ago LongCrafter: Towards Diverse Long-Context Understanding via Evidence-Graph-Guided Instruction Synthesis arXiv:2607.06160v1 Announce Type: new Abstract: Synthesizing long-context supervised fine-tuning (SFT) data is a scalable way to enhance the long-context understanding of large language models (LLMs), yet existing approaches share three limitations: narrow task coverage,… 10 Hugging Face Daily Papers research 3mo ago Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling Abstract Hierarchical Landmark Sparse Attention enables efficient long-context language modeling by learning chunk selection end-to-end, achieving performance comparable to full attention while extrapolating beyond training context lengths. Generated by… 35 Hugging Face Daily Papers research 3mo ago SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference Abstract SeKV introduces a resolution-adaptive semantic KV cache that compresses context into entropy-guided spans stored across GPU-CPU memory hierarchies, enabling efficient long-context processing with minimal memory overhead and preserved token-level detail. Generated by… 34 arXiv — Machine Learning research 3mo ago Training Hybrid Block Diffusion Language Models with Partial Bidirectionality arXiv:2607.02805v1 Announce Type: new Abstract: High-throughput long-context generation is one of the central challenges for large language models. Generation is typically memory-bandwidth-bound rather than compute-bound: each decoding step must stream the accumulated key/value… 32 arXiv — NLP / Computation & Language research 3mo ago Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling arXiv:2607.02980v1 Announce Type: new Abstract: Scaling modern large language models (LLMs) to long contexts is limited by the quadratic computation cost, and poor length extrapolation of dense attention. Chunk-wise sparse attention offers a promising alternative, but all… 36 arXiv — NLP / Computation & Language research 3mo ago SelfMem: Self-Optimizing Memory for AI Agents arXiv:2607.03726v1 Announce Type: new Abstract: While current AI agents support increasingly long context windows, tool use, and skill execution for long-horizon tasks, they still require memory systems to effectively leverage historical experience. Existing memory frameworks… 10 r/LocalLLaMA community 3mo ago Got my Ascent GX10 two days ago, ran REAP-pruned NVFP4 DeepSeek-V4-Flash on a single Spark, and it stays consistent at long context Got my Ascent GX10 two days ago and spent the last couple of days pushing a REAP-pruned NVFP4 DeepSeek-V4-Flash setup on a single Spark by patching the eugr/spark-vllm-docker image. Credit where it’s due: the REAPs were done by 0xSero . I’m just the person who wired it up,… 36 r/LocalLLaMA community 3mo ago Llama-Server is Throwing Away Your Perfectly Good KV Caches, and How to Fix It Introduction I watched 2.49 GB of state restore from disk in 1.23 seconds — and then get thrown away. llama-server's slot save/restore promises exactly what long-context work on budget hardware needs: park a session on disk, bring it back later without paying the prefill tax… 24 r/MachineLearning community 3mo ago Why I built a proactive context curator instead of a compactor — and what I got wrong for three months [P] Two ways to handle a context window that's filling up. Reactive: wait until it's full, then compact everything. Proactive: be picky about what gets added every turn so noise never piles up in the first place. Most coding agents take the reactive path. I spent months building the… 19 r/LocalLLaMA community 3mo ago I benchmarked 13 models at 65K-128K context to find out what actually matters for agentic workloads I benchmarked 13 models at 65K-128K context to find out what actually matters for agentic workloads — prefill dominates everything, and KV head count beats parameter count I've been running local LLMs for agentic workflows (tool use, coding agents, RAG) and kept seeing people… 36 r/LocalLLaMA community 3mo ago Appreciation post! Brought my first 3090 and been running Qwen 27B with 200K context, couldn't be happier. I'm using the club 3090 configuration and I highly recommend it! https://github.com/noonghunna/club-3090/tree/master Thanks to the community!   submitted by   /u/Top_Outlandishness78… 16 r/LocalLLaMA community 3mo ago We run 14 local-first agent harnesses with all working memory in TOON instead of Markdown — measured benchmarks (including the number that doesn't favor us) and the upstream bug report we got wrong We moved our agent fleet's working memory off Markdown and onto TOON (Token-Oriented Object Notation) in December 2025 and just wrote up what 14 harnesses taught us. The honest numbers (tiktoken o200k, 100 uniform CRM records): - TOON 2,068 tokens vs row-object JSON 3,074… 28 Hacker News — AI on Front Page community 3mo ago Google Books (or similar) all book scans – $200k bounty (2025) Article URL: https://software.annas-archive.gl/AnnaArchivist/annas-archive/-/work_items/234 Comments URL: https://news.ycombinator.com/item?id=48786838 Points: 201 # Comments: 95 12 r/LocalLLaMA community 3mo ago Comparing local inference speeds across a few real setups people are running (3090 vs 5090 vs dual 6000) Pulled together token rates from a few different local rigs people have reported running lately, just to get a sense of what's realistic at each hardware tier(source discord group) Qwen3.6 27B on a single 3090 (Q4/Q8 MTP, 128k ctx): ~50 tok/s inference, ~950 tok/s prompt… 29 r/MachineLearning community 3mo ago Proposal: Use semantic compression as input diffusion to read sessions larger than the context window [R] I've been trying to come up with a solution for keeping extremely long ai sessions coherent. Sometimes there is too much substance to risk compaction. With so much buzz around diffusion going on it got me thinking, what if we treat the context like a progressive render,… 7 r/LocalLLaMA community 3mo ago A fully local, self-hosted repo index for coding agents (Rust, MIT, runs offline) If you run local models for coding, feeding them repo context without blowing the context window is half the battle. I built basemind to index a repo locally and serve it over MCP: a code map across 300+ languages, git history and blame, and document RAG over 90+ formats, all on… 15 r/LocalLLaMA community 3mo ago Follow-up: DeepSeek V4 Flash on 2x RTX PRO 6000 finishes real coding tasks faster than Sonnet and Opus, at about Sonnet quality This is a follow-up to post about which local models stay fast deep into long context and I learned a lot from people here. I kept measuring after that and it turned into a proper indie coding bench. With DeepSeek V4 Flash running on vLLM it lands around Sonnet quality and it… 24 r/LocalLLaMA community 3mo ago Follow-up: GLM-5.2 NVFP4 on four DGX Sparks — the MTP mystery is solved, and it's now ~24 tok/s at 128K context Follow-up: GLM-5.2 NVFP4 on four DGX Sparks — the MTP mystery is solved, and it's now ~24 tok/s at 128K context This is a follow-up to my earlier post about running GLM-5.2 NVFP4 on 4x DGX Spark at 128K context. Short version of that post: 128K worked at ~15 tok/s with MTP1, and… 27 arXiv — NLP / Computation & Language research 3mo ago Multi-Head Recurrent Memory Agents arXiv:2607.01523v1 Announce Type: cross Abstract: Recurrent memory agents extend LLMs to arbitrarily long contexts by iteratively consolidating input into a fixed-size memory window. Despite their scalability, these agents exhibit a well-documented reliability problem:… 34 Hugging Face Daily Papers research 3mo ago Morphing into Hybrid Attention Models Abstract FlashMorph is an efficient layer selection method that formulates hybrid layer selection as a budget-constrained optimization problem, using morphable models and linearization regularization to improve long-context efficiency in Transformers. Generated by… 5 arXiv — Machine Learning research 3mo ago MosaicKV: Serving Long-Context LLM with Dynamic Two-D KV Cache Compression arXiv:2607.00760v1 Announce Type: new Abstract: Long-context LLM services now sustain prompts with hundreds of thousands to millions of tokens, making the key-value (KV) cache a first-order serving cost. Because the cache grows linearly with context length, it can exhaust GPU… 9 arXiv — Machine Learning research 3mo ago GSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache arXiv:2607.01065v1 Announce Type: new Abstract: The deployment of Large Language Models (LLMs) with extended context windows is increasingly constrained by the linear growth of Key-Value (KV) cache memory. Vector Quantization (VQ), particularly Residual Quantization (RQ), is a… 36 arXiv — NLP / Computation & Language research 3mo ago Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads arXiv:2607.01002v1 Announce Type: new Abstract: In long-context use, large language models frequently synthesize answers from the meaning of a relevant context span rather than literally copy-pasting them. Identifying which attention heads perform this synthesis matters for… 5 r/LocalLLaMA community 3mo ago Thinking about grabbing 4x Ascend GX10s Some in this sub have tested GLM5.2 on 4x DGX Sparks (or Ascend GX10) with 400-500 tok/s prompt processing and ~15 tok/s output at 128k context. Not blazing fast, but usable imo, especially with quantization. My thinking: If there's an open-source fable 5 sometime in december or… 20 arXiv — Machine Learning research 3mo ago Hierarchical Global Attention (HGA) arXiv:2606.30709v1 Announce Type: new Abstract: Hierarchical Global Attention (HGA) is a drop-in replacement for dense causal attention in pretrained long-context transformers. HGA preserves the original checkpoint parameters: the pretrained $W_Q$, $W_K$, $W_V$, and $W_O$… 23 arXiv — NLP / Computation & Language research 3mo ago RaBitQCache: Rotated Binary Quantization for KVCache in Long Context LLM Inference arXiv:2606.31519v1 Announce Type: cross Abstract: Long-context Large Language Model inference is severely bottlenecked by the massive Key-Value (KV) cache, yet existing sparse attention methods often suffer from static fixed-budget (Top-k) retrieval or rely on proxy scores that… 5 arXiv — Machine Learning research 3mo ago ECHO: Prune to act, trace to learn with selective turn memory in agentic RL arXiv:2606.31650v1 Announce Type: new Abstract: Long-horizon language agents must repeatedly interact with tools, accumulate evidence, and make decisions under bounded context windows. Existing context-management methods make such rollouts feasible by truncating distant history,… 12 arXiv — NLP / Computation & Language research 3mo ago SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference arXiv:2606.31145v1 Announce Type: new Abstract: Large language models increasingly operate over long contexts, where the KV cache becomes a dominant memory bottleneck: its size grows linearly with sequence length and must be retained throughout decoding, making full GPU caching… 11 r/LocalLLaMA community 3mo ago Devs - you have 64gb of VRAM - which model do you use for coding? I've currently settled on an unsloth version of Qwen 3.5 122b-a10b model (UD-IQ4_NL). With 100k bf16 context window, I only had to load a few layers into CPU/RAM, it runs around 30 tok/sec which is fine for me. I've tested many models, hours of testing but I am currently deeply… 32 r/LocalLLaMA community 3mo ago HydraHead: From Head-Level Functional Heterogeneity to Specialized Attention Hybridization (from the Qwen team) The quadratic complexity of attention poses a critical bottleneck for long-context processing, spurring interest in hybrid attention designs. Most open-source hybrid models adopt a layer-wise strategy. Yet, prior work has noted the inherent difficulty of integrating Linear… 9 Smol AI News news-outlet 3mo ago not much happened today **Anthropic** launched **Claude Sonnet 5** as its new default mid-tier frontier model, featuring a **1M-token context window**, enhanced agentic capabilities including planning, browser and terminal tool use, and autonomous execution previously requiring larger models. The model… 27 arXiv — Machine Learning research 3mo ago HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compression arXiv:2606.28831v1 Announce Type: new Abstract: Long-context LLM inference faces a fundamental conflict: head-adaptive compression algorithms (e.g., Top-$p$ nucleus sampling) offer superior accuracy by dynamically fluctuating memory budgets, yet modern inference engines (e.g.,… 29 arXiv — NLP / Computation & Language research 3mo ago Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution arXiv:2606.28548v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) have become a useful tool for extracting interpretable features in language models. However, standard SAE architectures operate on individual token activations, meaning that the number of active features… 25 arXiv — NLP / Computation & Language research 3mo ago Memory-Managed Long-Context Attention: A Preliminary Study of Editable Request-Local Memory arXiv:2606.28876v1 Announce Type: new Abstract: Long-context language models often conflate two different goals: compressing history into an efficient state, and maintaining reliable long-term memory. Linear, recurrent, and sparse attention reduce the cost of processing long… 14 arXiv — NLP / Computation & Language research 3mo ago Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM arXiv:2606.29563v1 Announce Type: new Abstract: Large language models (LLMs) excel at complex tasks like question answering and summarization, thanks to their ability to handle long-context inputs. However, deploying LLMs is costly, not only due to the high computational demands… 7 arXiv — NLP / Computation & Language research 3mo ago MATCH: Modulating Attention via In-Context Retrieval for Long-Context Transformers arXiv:2606.29844v1 Announce Type: new Abstract: The quadratic computational cost of traditional attention mechanisms poses a major bottleneck to the scalability and practical deployment of large language models (LLMs), particularly in long-context scenarios. To improve… 15 Page 7 of 10 · 500 articles ← Newer Older →
