News / #long-context Tag Long Context 500 articles archived under #long-context · RSS Sign in to follow Vercel — AI dev-tools 14d ago Ling 3.1 Flash is now available on AI Gateway Ling 3.1 Flash from InclusionAI is now available on AI Gateway. The model is free to use through October 13, 2026. Ling 3.1 Flash is a hybrid reasoning language model with 560B total parameters and 25B active per token. It has a 262K-token context window on AI Gateway. The model… 30 r/MachineLearning community 15d ago CoWindow and MassAlloc Attention: collective causal coverage and distribution-adaptive compute [R] I'm one of the authors of two recent papers exploring different sources of redundant computation in attention. I'd like to share the ideas and hear feedback from people working on long-context models and attention kernels. CoWindow Attention (CoWA) distributes distant context… 26 arXiv — Machine Learning research 15d ago CacheReforge: Bounded Recovery for Stale KV Caches under Evolving Adapters arXiv:2609.30884v1 Announce Type: new Abstract: Large language models rely on KV caching to reduce repeated prefill computation in long context and interactive applications. As lightweight adapters evolve, cached states reflect earlier versions, so stale reuse distorts current… 32 arXiv — Machine Learning research 15d ago Block Sparse Attention with Log-Linear Complexity arXiv:2609.31093v1 Announce Type: new Abstract: Scaling language models to long contexts is limited by the quadratic cost of self-attention. Block sparse attention offers an efficient alternative, but selecting the retained blocks remains a bottleneck. Conventional block… 23 arXiv — NLP / Computation & Language research 16d ago MoSAR: Mixture of Semantic Attention Regimes for Learning Adaptive and Approximable Attention Geometries arXiv:2609.31261v1 Announce Type: new Abstract: The quadratic complexity of dense self-attention remains a central bottleneck for long-context language modeling. Many efficient alternatives address this cost by deciding in advance where attention should be sparse or local. We… 13 arXiv — NLP / Computation & Language research 16d ago Highlight-Then-Summarize: Learning to Compress Evidence for Long-Context Understanding arXiv:2609.31382v1 Announce Type: new Abstract: Long-context understanding requires large language models (LLMs) to reason over lengthy documents, conversations, and code, yet task-relevant evidence is often sparse and scattered amid substantial irrelevant and redundant content.… 26 arXiv — NLP / Computation & Language research 16d ago UniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic Sparsification arXiv:2605.06221v2 Announce Type: replace Abstract: As large language models (LLMs) continue to advance rapidly, they are becoming increasingly capable while simultaneously demanding ever-longer context lengths. To improve the inference efficiency of long-context processing,… 30 r/LocalLLaMA community 17d ago Naive-N0.5-Flash - 309B-A15.5B https://naive.ai/en/research/ Built for coding and AI R&D 1M context Hybrid SWA/DSA   submitted by   /u/nullmove [link]   [comments] 35 arXiv — NLP / Computation & Language research 19d ago ChunkRank: Model-Aware Text Chunking and Abstention-Aware Answer Selection for LLM Pipelines arXiv:2609.29828v1 Announce Type: new Abstract: We present ChunkRank, an open-source Python library that derives chunk boundaries from a target model's tokenizer and context window, and selects an answer among candidates produced independently per chunk. It ships a validated… 38 r/LocalLLaMA community 20d ago model : add Ling 3.0 VL support by aetherbird · Pull Request #29151 · ggml-org/llama.cpp Model Overview Ling-3.0-flash-VL inherits the language, reasoning, and long-context capabilities of Ling-3.0-flash, while extending them with native image and video understanding. The model has 124B total parameters, with only 5.5B parameters activated per token, and supports a… 31 arXiv — NLP / Computation & Language research 20d ago When Learned Context Planning Fails to Beat Strong Retrieval: A Controlled Study of Planning, Routing, and Reranking for Long-Context QA arXiv:2609.26976v1 Announce Type: new Abstract: Learned context planning selects evidence atoms before an answer model reasons over them. We test whether this learned selection improves long-context multiple-choice QA after strong retrieval, routing, budgeted-selector, and… 27 arXiv — NLP / Computation & Language research 20d ago MWE-ECL: Recoverable Long-Range Context Does Not Always Override Local Lexical Priors arXiv:2609.27590v1 Announce Type: new Abstract: Long-context evaluations often test whether a model can recover distant evidence, but recoverability does not guarantee behavioral influence. We test the prediction that a distant discourse anchor can remain explicitly recoverable… 21 arXiv — NLP / Computation & Language research 20d ago DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Video arXiv:2609.27470v1 Announce Type: cross Abstract: Recent video-language models increasingly adopt hybrid architectures that interleave linear and full attention layers for efficient long-context processing. While the recurrent state of linear attention remains fixed in size, the… 9 arXiv — Machine Learning research 21d ago CompKV: Compensation-Aware KV Selection for Long-Context LLM Inference arXiv:2609.26300v1 Announce Type: new Abstract: Despite their strong performance, large language models (LLMs) are bottlenecked by KV cache memory traffic during long-context inference. Sparse attention is widely used to accelerate LLM inference by computing exact attention over… 10 arXiv — NLP / Computation & Language research 21d ago Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference arXiv:2609.25537v1 Announce Type: new Abstract: Large language model (LLM) inference is constrained by the quadratic scaling of self-attention and the linear scaling of the KV cache, increasing latency, energy consumption, and GPU memory demand as context length scales. Existing… 24 arXiv — NLP / Computation & Language research 21d ago HySparse2: Hybrid Sparse Attention with Two-Level KV Sharing arXiv:2609.26368v1 Announce Type: new Abstract: Long-horizon and multi-turn agents typically generate short actions and process long observations from tools and environments. This growing context demands efficient prefill, compact KV-cache storage, and accurate long-context… 38 r/LocalLLaMA community 22d ago Is llama.cpp meant to be slow at long context, even when you aren't using that context? I am trying out a few fine-tunes of Qwen 3.5 9B @ IQ4_XS @ 131K context and trying to go mostly local (free beats cheap, after all). However, it is much slower than at, say 16K context, even when I am not actually using 131K tokens in the first place. Anyone know why this is? I… 30 arXiv — NLP / Computation & Language research 22d ago Context Poisoning as Extreme-Value Attention Interference in Long-Context Language Models arXiv:2609.22101v1 Announce Type: new Abstract: Large language models can process increasingly long prompts, yet their ability to locate and use decisive evidence may degrade as irrelevant or confusable context is added. We formulate this phenomenon, which we call context… 8 arXiv — NLP / Computation & Language research 22d ago Vox-Infinity: Benchmarking the Limits of Long-Context Spoken Language Models arXiv:2609.22452v1 Announce Type: new Abstract: Long-context understanding remains a fundamental challenge for large language models, as excessively long inputs often lead models to forget salient information. This issue is even more pronounced in the speech domain, where audio,… 33 arXiv — NLP / Computation & Language research 22d ago Block-Sparse Attention with Semantic-Geometric Decoupled Routing arXiv:2609.22884v1 Announce Type: new Abstract: Long-context inference has become a defining capability of large language models, but exact dense attention remains costly due to its quadratic scaling with sequence length. Block-sparse attention offers a hardware-friendly… 36 arXiv — Machine Learning research 23d ago Elastic Threshold Attention: Learned Contextual Sparsity for Long-Context Decoding arXiv:2609.20888v1 Announce Type: new Abstract: Massive KV caches can cause severe memory-bandwidth bottlenecks during long-context decoding. Sparse attention methods mitigate this via selective loading, but that comes at a cost: rigid heuristics drop necessary context, leading… 38 arXiv — Machine Learning research 23d ago TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching arXiv:2609.21172v1 Announce Type: new Abstract: Large language models (LLMs) are moving onto mobile devices for increasingly diverse workloads over text, images, video, and audio. These applications often require long contexts, making the Key-Value (KV) cache a dominant memory… 28 Vercel — AI dev-tools 23d ago Grok 4.7 now available and 40% off on AI Gateway, fx, and eve Grok 4.7 from SpaceXAI is now available on AI Gateway and 40% off through September 27. The discount applies automatically when you call spacexai/grok-4.7 . Grok 4.7 has a 500K token context window and supports low, medium, high, and xhigh reasoning levels, giving you control… 31 r/LocalLLaMA community 25d ago Qwen3.8-Flash-Next at 1M context on Strix Halo: 38 tok/s decode, 18 min prefill (halogen 0.12.0) Hi. I saw some feedback that halogen was degrading at context depth. So I fixed that. Served through the image, same machine, same session, same prompts, 0.11.10 vs 0.12.0: decode at 1,004,581 tokens of context: 27.3 to 38.3 tok/s (default speculative drafter) decode at 258,794:… 18 The Information — AI news-outlet 26d ago Communicate Technical Topics to a Non-Technical Audience with Google Gemini In a room full of IT professionals, words and phrases like “tokenization,” “retrieval-augmented generation,” and “context window” need no explanation. But when technology leaders present to their CEOs and other company leaders about topics like AI, they often find that they need… 10 ElevenLabs official-blog 26d ago What is a context window? What every LLM user should know Learn what is a context window is, how LLMs process tokens, why larger context windows affect performance, and how to manage context in AI applications. 25 arXiv — Machine Learning research 26d ago Block Parallelism For Efficient Distributed Long-Context Diffusion Language Model Training arXiv:2609.19242v1 Announce Type: new Abstract: Block diffusion language models (BDLMs) combine autoregressive dependencies across blocks with parallel denoising within blocks, but long-context training is constrained by distributed attention communication and activation memory.… 30 arXiv — NLP / Computation & Language research 26d ago DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression arXiv:2609.19969v1 Announce Type: new Abstract: The widespread adoption of long-horizon agents has made model workloads increasingly input-heavy. Although prior work has substantially reduced the cost of long-context computation, prefill remains computationally expensive, and… 36 arXiv — NLP / Computation & Language research 26d ago On-Demand Attention: Language Models Know When to Recall arXiv:2609.20734v1 Announce Type: new Abstract: Reasoning and agentic workloads increasingly demand efficient long-context inference. Yet full-attention decoding reads the growing history at every step, regardless of its benefit to the next prediction. We show that a pretrained… 30 Hugging Face Daily Papers research 27d ago Flattening Every Memory Peak in Long-Context Mixture-of-Experts Training Abstract Training a Mixture-of-Experts (MoE) model at long context or large batch size fails as soon as any one component's peak allocation exceeds device memory, so the target is every peak at once, not the average footprint. Four are left unbounded by the parallelism plans in… 5 Hugging Face Daily Papers research 27d ago SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization Abstract Recently, linear attention layers have been increasingly adopted to replace softmax attention at scale for long-context modeling. However, existing context extension approaches typically apply continued pretraining directly without modifying these layers, overlooking… 17 arXiv — Machine Learning research 27d ago Beyond Static RAG: An Adaptive, Tri-Metric Routing Framework for Efficient Long-Context Inference on Commodity GPUs arXiv:2609.17564v1 Announce Type: new Abstract: Deploying retrieval-augmented generation (RAG) on commodity GPUs such as the NVIDIA T4 (16 GB VRAM) exposes a practical failure mode we call the Compression Paradox: neural prompt compression can add key-value (KV) cache contention… 7 arXiv — NLP / Computation & Language research 27d ago Long-Context Demonstration Selection Using State Space Models arXiv:2609.17888v1 Announce Type: cross Abstract: We study the problem of demonstration selection, which involves selecting a subset of examples for prepending to a query to a language model. This problem is closely related to in-context learning and language model inference.… 17 arXiv — NLP / Computation & Language research 27d ago ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference arXiv:2609.17943v1 Announce Type: cross Abstract: Long-context LLM inference is bottlenecked by attention, whose repeated KV-cache reads make decoding memory-bound. Self-speculative decoding alleviates this by drafting tokens with sparse attention and verifying them with full… 8 r/LocalLLaMA community 28d ago Qwen 3.8 27B Running for 63 hours on a RTX 3090 to solve the Riemann hypothesis I let Qwen 3.8 27B 4bit quantized with 100K context window run autonomously for 63 hours (50 million+ tokens) to try to solve the RH. Of course it did not solve it, but the experiment still shows it's internal work, memory organization, strategies used and more. The interesting… 11 arXiv — NLP / Computation & Language research 28d ago Can LLMs Follow the Pulse of a Crisis? Evaluating Crisis Sentiment in Bangladesh's July Uprising arXiv:2609.16997v1 Announce Type: new Abstract: Crisis sentiment analysis is especially challenging for low-resource languages such as Bangla, where language, context, and public reaction shift rapidly. We introduce UNRESTSENT200K, a Bangla crisis sentiment dataset with… 8 arXiv — NLP / Computation & Language research 28d ago Where Should a Document Live: Context, Representations, or Parameters? arXiv:2609.17346v1 Announce Type: new Abstract: To answer questions outside of their pre-training data, large language models (LLMs) need access to new information, which can be presented in the context window as documents, encoded into the model's parameters, or injected as… 21 arXiv — NLP / Computation & Language research 28d ago VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs arXiv:2609.16722v1 Announce Type: cross Abstract: Scaling Multimodal Large Language Models (MLLMs) to long-form video understanding is bottlenecked by the explosion of visual tokens, which saturates context windows and incurs prohibitive costs. Current solutions predominantly… 28 r/LocalLLaMA community 29d ago Qwen3.8-27B-NVFP4 1M context. So far so good. I am a beginner, Took a while to get started, get everything right. This setup is native not container. Still not sure if I did this right, or if I can tune this more. Environment=HF_HUB_OFFLINE=1 Environment=VLLM_LOGGING_LEVEL=INFO Environment=VLLM_ALLOW_LONG_MAX_MODEL_LEN=1… 15 arXiv — Machine Learning research 29d ago Tabby: An Open Pretraining Recipe for Time Series Foundation Models arXiv:2609.13956v1 Announce Type: new Abstract: In this report, we release Tabby, a long context probabilistic time series foundation model, together with a complete and open recipe of how it was built. Tabby adopts an encoder-only patch Transformer architecture and concentrates… 35 arXiv — NLP / Computation & Language research 29d ago SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization arXiv:2609.14320v1 Announce Type: new Abstract: Recently, linear attention layers have been increasingly adopted to replace softmax attention at scale for long-context modeling. However, existing context extension approaches typically apply continued pretraining directly without… 26 Hugging Face Daily Papers research 29d ago ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search Abstract ZGCM-1 is a 7B open foundation model that combines internal reasoning with external tool use, trained via efficient architecture-system co-design, progressive long-context scaling, and autonomous agent workflows to achieve strong reasoning and efficiency. Generated by… 26 arXiv — NLP / Computation & Language research 1mo ago Doc2FRC: Length-Consistent Document-Level Machine Translation via Fixed-Range Chunking arXiv:2609.12674v1 Announce Type: new Abstract: Advanced large language models (LLMs) with long context windows can substantially reduce input truncation in document-level machine translation (DocMT). However, direct Doc2Doc translation remains prone to n-gram repetition and… 31 arXiv — NLP / Computation & Language research 1mo ago Residual Vector-based Reconstruction as Long-Context Recall Regardless of Context Window Size arXiv:2609.12686v1 Announce Type: cross Abstract: Large language models (LLMs) process long contexts, including long documents and lengthy conversations, but face token-level memory usage that increases proportionally to input length. Although model optimization and lossy prompt… 35 r/LocalLLaMA community 1mo ago This draft model is OP on 16 GB cards for Qwen 3.8 27b https://huggingface.co/HermiHg/Qwen3.8-27B-DFlash2-Q2_K_S-MIX-GGUF I used this draft model with https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF with the IQ3_XXS with 128k context and I saw it averaging about 60 tokens per second tg speed on the 16 GB RX 9070 XT. This… 16 r/LocalLLaMA community 1mo ago Agnes-AI/Agnes-3.0-Flash 33B Multimodal, AA score: 36 I find this new model at HF: "Built for demanding work. A 262 144-token context window, adjustable reasoning effort, tool calling, and text, image and video understanding. Architecture Agnes-3.0-Flash is a hybrid-attention decoder: three of every four layers run a gated delta… 26 arXiv — NLP / Computation & Language research 1mo ago Larger Context Window, Fewer Overcorrections: Optimizing Prompts and Batching for Minimal-Edit Grammatical Error Correction arXiv:2609.10810v1 Announce Type: new Abstract: Minimal-edit Grammatical Error Correction (GEC) is a challenging task for zero- and few-shot prompted Large Language Models (LLMs), which systematically overcorrect and degrade $F_{0.5}$ by rewriting well-formed spans. While… 38 r/LocalLLaMA community 1mo ago GigaChat-3.5-Reasoning Hey y'all! We've released a new model in our lineup: GigaChat-3.5 Reasoning. It's a 432B-A28B MoE with Gated DeltaNet for long-context efficiency. We trained domain experts (code, math, general, etc.) with CISPO and then distilled them into a single model via on-policy… 18 arXiv — NLP / Computation & Language research 1mo ago ConvMem: Convolutional Memory for Long-Context Reasoning arXiv:2609.10441v1 Announce Type: cross Abstract: While Large Language Models (LLMs) have demonstrated impressive capabilities, they often struggle with extremely long contexts due to fixed context limits. To address this, sequential approaches like MemAgent extend the effective… 38 r/LocalLLaMA community 1mo ago GLM 5.3 Flash Q4 @ 60tps / 550tps on M3 Ultra I have been a dwarfstar fan for awhile and I really liked glm 5.3 flash but needed it to be materially faster to feel good using it. In the screenshot you can see the outcome of using the model with a claude code harness at ~200k depth, with many tool calls and averaging over… 29 Page 2 of 10 · 500 articles ← Newer Older →
