News / #long-context Tag Long Context 433 articles archived under #long-context · RSS Sign in to follow arXiv — NLP / Computation & Language research 26d ago Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation arXiv:2608.01676v1 Announce Type: new Abstract: Sparse attention is widely deployed in long-context serving stacks, yet no framework audits how discarding blocks changes the influence of specific content on model output. We first establish that the phenomenon is real and causal:… 18 arXiv — NLP / Computation & Language research 27d ago ThinkReset: Learnable Intermediate Interface Construction for Bounded-Context Long-Horizon Reasoning arXiv:2607.28642v1 Announce Type: cross Abstract: Long chain-of-thought reasoning improves performance on complex problems, but it also introduces redundancy accumulation, context overflow, and error anchoring. We argue that under bounded context windows, the core bottleneck is… 31 arXiv — NLP / Computation & Language research 27d ago Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements arXiv:2607.28661v1 Announce Type: new Abstract: Do Large Language Models (LLMs) possess genuine structural reasoning, or merely rely on surface-level pattern matching? The financial domain, demanding numerical precision and multi-step logic over long contexts, is an ideal… 30 arXiv — NLP / Computation & Language research 27d ago ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression arXiv:2607.29591v1 Announce Type: new Abstract: KV cache compression is essential for efficient long-context inference. Existing eviction methods permanently discard unselected tokens and consequently remove their aggregate contribution to attention. Merging-based alternatives… 10 Vercel — AI dev-tools 28d ago Qwen 3.8 Max now available on Vercel AI Gateway Qwen 3.8 Max is now available on AI Gateway. Qwen 3.8 Max handles text-only and vision-language work in one model, with 2.4 trillion parameters and a context window of up to 1 million tokens. The model is suited for software engineering and office productivity, along with visual… 13 r/LocalLLaMA community 28d ago LongCat-Flash-Lite-Sparse Is Now Available for Download The weights have now been added to the repo an hour ago. This model is built upon LongCat-Flash-Lite , the differences are that LongCat-Flash-Lite-Sparse : Replaces dense MLA with LongCat Sparse Attention (LSA) Natively supports context lengths of up to 1M tokens (vs 256k for… 13 r/LocalLLaMA community 29d ago What speeds are everyone getting with deepseek v4 flash 0731? What speeds are everyone getting with deepseek v4 flash 0731? I’m getting~200 tps prompt processing / ~11 tps token gen, on 4x5060ti16gb with ddr4 3200 ram at 4-channel, via llamacpp, with context window of 128000, -ub/-b at 4096, “q8” unsloth’s lossless quant   submitted by… 36 NVIDIA Developer Blog official-blog 29d ago Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because... 20 llama.cpp releases dev-tools 1mo ago b10201 ggml-webgpu: improve flash_attn_vec for quantized KV at long contexts ( #25956 ) improve fa of quantized kv cache Fix some bugs and some comments. fix v type check and some comments Fix build error caused by rebasing editorconfig checking pass Website: https://llama.app… 13 arXiv — Machine Learning research 1mo ago Beyond KV Reconstruction: Functional Reconstruction for MLA Draft Models in Speculative Decoding arXiv:2607.27269v1 Announce Type: new Abstract: Multi-head latent attention (MLA) is increasingly important for long-context LLM inference because compact latent states replace the growing key-value (KV) cache and reduce decoding memory traffic. Yet most capable open checkpoints… 16 arXiv — NLP / Computation & Language research 1mo ago Recall Before You Rank: Similarity-Guided Top-$K$ Reuse for Efficient Long-Context Attention arXiv:2607.27692v1 Announce Type: new Abstract: Top-$K$ sparse attention reduces the cost of Softmax and value aggregation by attending to only a small subset of key--value (KV) entries. However, identifying this subset still requires scoring the current query against the full… 13 arXiv — NLP / Computation & Language research 1mo ago PCAP-LM: An LLM-Native Text Representation for TLS Bulk Traffic Analysis arXiv:2607.28100v1 Announce Type: cross Abstract: Large language models (LLMs) offer powerful reasoning capabilities for network traffic analysis, but standard capture formats and their textual equivalents are prohibitively verbose, overflowing LLM context windows by two orders… 37 r/LocalLLaMA community 1mo ago Inkling-Small by thinkingmachines 276B total parameters, 12B active, 1M context window. Blog post: https://thinkingmachines.ai/news/inkling-small/ NVFP4: https://huggingface.co/thinkingmachines/Inkling-Small-NVFP4 GGUF's by Unsloth: https://huggingface.co/unsloth/Inkling-Small-GGUF   submitted by  … 20 arXiv — NLP / Computation & Language research 1mo ago Mergeable Model-Side Aggregation States for Long-Context Language Models arXiv:2607.26448v1 Announce Type: new Abstract: A known limitation of long-context language models is their increasingly unreliable performance in non-additive, set-based aggregation as context length grows. Examples include cardinality estimation, set relationships, and grouped… 22 arXiv — NLP / Computation & Language research 1mo ago DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search arXiv:2607.27178v1 Announce Type: new Abstract: State-of-the-art retrieval models increasingly rely on closed training data, creating a reproducibility gap. We present an open end-to-end recipe for training retrieval models and study how English supervision transfers to… 21 arXiv — NLP / Computation & Language research 1mo ago MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent arXiv:2507.02259v2 Announce Type: replace Abstract: Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents with linear complexity without performance degradation during extrapolation remains the ultimate challenge… 22 Vercel — AI dev-tools 1mo ago AI Gateway: GPT-5.6 pricing and speed updates On AI Gateway , GPT-5.6 Luna and GPT-5.6 Terra are now cheaper and GPT-5.6 Sol is faster. AI Gateway adds no markup on token pricing, so these changes reach you at the upstream rate. The changes apply to both short and long context pricing. Model Change Input: Short context (per… 4 arXiv — NLP / Computation & Language research 1mo ago CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention arXiv:2607.25291v1 Announce Type: new Abstract: The quadratic cost of self-attention makes long-context inference prohibitively expensive, and proxy-based block-sparse attention has become a practical remedy. Existing methods typically rely on a proxy to predict a binary sparse… 32 arXiv — NLP / Computation & Language research 1mo ago GLIDE: Guided Layerwise Hybrid Attention for Efficient LLM Inference arXiv:2607.24788v1 Announce Type: cross Abstract: As Large Language Models scale to increasingly long contexts, the memory I/O and computational overhead of the Key-Value (KV) cache during decoding emerges as the primary throughput bottleneck. To address this, we propose GLIDE,… 36 arXiv — NLP / Computation & Language research 1mo ago Addressable Recall Compaction for Long Context-Window Control in AI Agents arXiv:2607.25066v1 Announce Type: cross Abstract: Long-horizon LLM agents accumulate reasoning traces, actions, and tool observations that can eventually exceed a model's fixed context window. Existing compaction methods address this limitation by discarding, summarizing, or… 30 arXiv — NLP / Computation & Language research 1mo ago HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following arXiv:2607.25398v1 Announce Type: cross Abstract: Language-model agents are increasingly deployed under standing instructions: a system prompt, a policy file, or a skills document is placed in context, and the agent is trusted to let it govern every action that follows. Existing… 6 Hugging Face official-blog 1mo ago LFM2.5-Encoders for Fast Long-Context Inference on CPU Back to Articles a]:hidden"> LFM2.5-Encoders for Fast Long-Context Inference on CPU Team Article Published July 28, 2026 Upvote 14 Fernando Fernandes Neto fernandofernandes LiquidAI Edoardo Mosca EdoardoMosca LiquidAI Maxime Labonne mlabonne LiquidAI Leonie Monigatti iamleonie… 18 arXiv — Machine Learning research 1mo ago Variational-Ising-Attention (VIA):TailoredAttentionMattersfor Science arXiv:2607.23634v1 Announce Type: new Abstract: Attention enables context modeling via query-key scoring with softmax normalization. Driven by industrial long-context demands, mainstream research has converged toward sparsity and efficiency--yet softmax's independence assumption… 36 arXiv — NLP / Computation & Language research 1mo ago INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models arXiv:2607.24273v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown strong potential in financial reasoning, but existing benchmarks often evaluate domain knowledge, numerical reasoning, long-context understanding, and tool use in separate settings. This… 32 arXiv — NLP / Computation & Language research 1mo ago Kimi K3: Open Frontier Intelligence arXiv:2607.24653v1 Announce Type: new Abstract: We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention… 20 Hugging Face Daily Papers research 1mo ago Kimi K3: Open Frontier Intelligence Abstract We introduce Kimi K3, a 2.8T parameter Mixture-of-Experts model with 104 billion activated parameters, native vision capabilities, and a 1-million-token context window. Kimi K3 is built on Kimi Delta Attention and Attention Residuals, which improve information flow… 7 r/LocalLLaMA community 1mo ago First evidence of a pending qwen3.7 open weights release. Qwen3.7-flash is on open router. They referred to Qwen3.6-35b-a3b as Qwen3.6 flash so this is likely a small MoE. The prices are substantially cheaper than 3.6 flash with a native 1M context window.   submitted by   /u/fulgencio_batista [link]   [comments] 37 arXiv — NLP / Computation & Language research 1mo ago Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets arXiv:2607.21692v1 Announce Type: cross Abstract: Sparse attention reduces the cost of long contexts by allowing each query to read only selected parts of the input. These selectors are often trained by distilling the attention patterns of a dense teacher, assuming that… 31 arXiv — Machine Learning research 1mo ago RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention arXiv:2607.21927v1 Announce Type: new Abstract: Full self-attention in large language models scales as O(N^2), which limits long-context document analysis to 65,536 tokens and requires costly GPU clusters. The Reduced Interaction Sampling (RIS) inference engine addresses this… 31 r/LocalLLaMA community 1mo ago Mobile Offline LLMs: What do you use them for? I've spent the last year or so playing around with open source MLX and GGUF models on iPhone hardware. Given the limitations in memory, GPU/CPU/ANE, and in turn the context window I've been trying to figure out the best use cases for them. I've also done a lot of testing with… 13 r/LocalLLaMA community 1mo ago Kimi Linear 48B A3B? Just noticed this exists, 1M context MOE with 48B par seems just like what Ive been looking for - it runs pretty damn fast too compared to Qwen 3.6 35B. after some testing it seems capable of producing *not terrible* results but it always tries to go for the minimun possible… 24 r/LocalLLaMA community 1mo ago My GX10 died Everything ran fine, I was using UD 3.6 Q6 for 35 and 27B, each 4 concurrent requests at 200K context. I had Dify and Mastra to play around with, Unsloth studio to get around to and vLLM ready for whenever I decided to do some more testing. LLama-swap above lama.cpp and liteLLM… 30 r/LocalLLaMA community 1mo ago DKV: Open-source KV-cache compression framework for local LLM inference (CLI + technical report) Hi everyone! Over the past five months I've been working on DKV (DifferentialKV), an open-source project exploring KV-cache compression for long-context local LLM inference. The goal is to reduce KV-cache memory requirements through anchor-based representations, joint low-rank… 21 arXiv — Machine Learning research 1mo ago Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches arXiv:2607.20538v1 Announce Type: new Abstract: Long-context Transformer inference increasingly relies on KV-cache compression or quantization. Prior rotation and transform-coding results suggest that the channel basis of each key/value vector affects how faithfully a fixed… 14 arXiv — NLP / Computation & Language research 1mo ago news-crawler-LM: A Small Long-Context Model For High-Quality News Crawling arXiv:2607.21284v1 Announce Type: new Abstract: Extracting structured content from news pages remains challenging due to heterogeneous HTML layouts, inconsistent markup, and substantial boilerplate such as navigation elements and advertisements. Rule-based news crawlers can… 13 Vercel — AI dev-tools 1mo ago Ling 3.0 Flash is now available on AI Gateway Ling 3.0 Flash from Ant Group is now available on AI Gateway. The model is free to use for the next three weeks, through August 3rd. Ling 3.0 Flash is a Mixture-of-Experts model with 124B total parameters and about 5.1B active per token. It has a 256K token context window and… 12 r/LocalLLaMA community 1mo ago Tokenizer Expansion: Upgrading a Model's Tokenizer in Place - LFM2.5-8B-A1B Today, we're sharing the recipe behind the new tokenizer in LFM2.5-8B-A1B . It upgrades a pre-trained model's tokenizer in place , without retraining from scratch. We doubled the vocabulary from 65K to 128K to fix the languages our original tokenizer split too finely. Blog:… 34 arXiv — NLP / Computation & Language research 1mo ago Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning arXiv:2607.19345v1 Announce Type: new Abstract: Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier. However, we identify a critical… 16 arXiv — Machine Learning research 1mo ago High-accuracy Low-Bit KV-Cache Quantization via Local Distribution Restoration arXiv:2607.16248v1 Announce Type: new Abstract: Long-context large language model inference relies on the KV cache to avoid redundant attention computation, but incurs high memory and bandwidth overheads. Low-bit KV-cache quantization reduces this cost, yet it severely degrade… 17 arXiv — Machine Learning research 1mo ago More Than Memory: Task-Conditioned Signed FFN Writes in Long-Context Retrieval arXiv:2607.16254v1 Announce Type: new Abstract: FFNs are often treated as parametric memories. In long-context retrieval, however, the sharper question is not only what they store, but whether their native residual writes push the current retrieval state toward or away from the… 10 arXiv — NLP / Computation & Language research 1mo ago C$^2$KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference arXiv:2607.17715v1 Announce Type: new Abstract: Long-context inference is central to modern large language model (LLM) applications such as retrieval-augmented generation and multi-document reasoning. To mitigate the growing inference cost, recent work has explored key-value… 20 arXiv — NLP / Computation & Language research 1mo ago SWE-Pruner Pro: The Coder LLM Already Knows What to Prune arXiv:2607.18213v1 Announce Type: new Abstract: Pruning long context for coding agents has been a vital technology for efficient context management. While existing context pruning methods such as SWE-Pruner realize this by attaching a separate code classifier, we find the agent… 8 arXiv — NLP / Computation & Language research 1mo ago Is Progressive Disclosure All You Need for Long-Context Agents? arXiv:2607.17598v1 Announce Type: cross Abstract: Long-document question answering usually forces a choice between loading the whole document into the context window and bolting on a separate retriever. Agentic AI suggests a broader option, giving the agent the document path and… 30 Hugging Face Daily Papers research 1mo ago SWE-Pruner Pro: The Coder LLM Already Knows What to Prune Abstract Pruning long context for coding agents has been a vital technology for efficient context management. While existing context pruning methods such as SWE-Pruner realize this by attaching a separate code classifier, we find the agent itself encodes internal representations… 22 Vercel — AI dev-tools 1mo ago Laguna S 2.1 is now available on AI Gateway Laguna S 2.1 from Poolside is now available on AI Gateway. There are 2 versions of the model available: Free version (256K context window): poolside/laguna-s-2.1-free Paid version (1M context window): poolside/laguna-s-2.1 Laguna S 2.1 is an open-weight Mixture-of-Experts model… 17 arXiv — NLP / Computation & Language research 1mo ago VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs arXiv:2607.15498v1 Announce Type: new Abstract: The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference. Two leading training-free families are both structurally limited: token-selection methods (SnapKV, Ada-KV) score… 31 r/LocalLLaMA community 1mo ago Fractale-350M-base: memory as trained behaviour instead of long context, a fully open research release Some of you may remember my post about the research project behind this: a trained fast-weight memory, with the paper and the full research log at github.com/kkuette/thought-bank. This is the follow-up. The first public model of the series is out. Quick context: solo researcher,… 11 Hugging Face Daily Papers research 1mo ago LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget Abstract A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment. The gap is especially… 16 arXiv — Machine Learning research 1mo ago LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget arXiv:2607.14952v1 Announce Type: new Abstract: A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization… 14 arXiv — NLP / Computation & Language research 1mo ago PReM: Learning What to Preserve and When to Refresh for Context Compression arXiv:2607.14327v1 Announce Type: new Abstract: Efficient long-context inference is not only about reducing memory cost, but also about keeping useful contextual evidence accessible as generation proceeds. However, existing compression-oriented approaches, such as key-value (KV)… 32 Page 3 of 9 · 433 articles ← Newer Older →
