News / #gpu Tag Gpu 500 articles archived under #gpu · RSS Sign in to follow arXiv — NLP / Computation & Language research 19d ago CORDIAL: Calibrating Ordinal LLM Outputs from Few Labels arXiv:2609.29807v1 Announce Type: new Abstract: A large language model (LLM) can turn a text into a distribution over an ordered scale, but that distribution is a noisy measurement: saturated, compressed or exaggerated, and biased in a consistent direction. We propose CORDIAL,… 23 arXiv — NLP / Computation & Language research 19d ago Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs arXiv:2609.29845v1 Announce Type: new Abstract: While Large Language Models (LLMs) rely on highly non-linear components, in this work we demonstrate that they exhibit fundamental linearity: when inputs from distinct text streams are linearly combined, the model outputs a… 24 arXiv — NLP / Computation & Language research 19d ago Encoded but Not Decoded: Layer-Localized Evidence for a Three-Level Gap in LLM Syntax arXiv:2609.29848v1 Announce Type: new Abstract: A language model can fail a syntactic test in two distinct ways: by not encoding the relevant structure, or by encoding it but failing to use it at the output. Behavioral evaluation alone cannot tell these apart. We propose a… 13 arXiv — NLP / Computation & Language research 19d ago JevOut: Natural Context Can Flip Decision Models arXiv:2609.30243v1 Announce Type: new Abstract: Dedicated decision models such as Jev map unstructured language to probability distributions over finite choices, allowing their outputs to directly route requests, select tools, and trigger actions. Yet real-world inputs rarely… 22 r/LocalLLaMA community 20d ago M5 Ultra 80Core GLM-5.3-Flash on DwarfStar Speeds I've been playing around with various models on the M5 Ultra 256GB 80-core Mac Studio. These are the results over many rounds of agentic inferencing. I'm happy with the performance. Glad to have the large amount of RAM. But it does feel like the GPU is underpowered for this… 6 llama.cpp releases dev-tools 20d ago b11166 cuda : add F16 kernel support for CONV_2D_DW ( #29064 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49985945 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS… 12 The Information — AI news-outlet 20d ago Former OpenAI Data Center Chief Is Now At Nvidia Chris Malone, OpenAI’s former head of data centers, joined Nvidia as the vice president of Nvidia’s DSX Platform this month, according to his LinkedIn profile. DSX is the Nvidia division that helps customers design and build AI data centers according to Nvidia’s specifications.… 4 r/LocalLLaMA community 20d ago Qwen3.8-Flash-Next on 12GB VRAM - 65 tokens per second A while ago I posted 15 tok/s output and 100-120 tok/s prompt processing with the IQ3_XXS quant on a 12GB RTX 5070 using llama.cpp. Since then I built my own inference engine for this one model and this kind of PC. The same IQ3_XXS now runs at ~65 tok/s output and ~430 tok/s… 38 Ars Technica — AI news-outlet 20d ago Google's first Suncatcher orbital data center test launches October 1 Google's experimental orbital data center will have four TPUs and only run for 15 minutes at a time. 21 The Information — AI news-outlet 20d ago Are GPU Loans Safe From Rising Rates? The Federal Reserve’s recent rate hike, and rising expectations of another soon, put a fresh spotlight on financing arrangements for the wide array of companies borrowing money to pay for AI data centers and chips. Rates are rising just as investors have been growing more… 5 The Information — AI news-outlet 20d ago What I Learned From AI Agenda Live What a day! I loved meeting many of our amazing subscribers yesterday at our AI Agenda Live conference, where my colleagues and I interviewed leaders from companies like OpenAI , Google DeepMind , Nvidia , Blackstone , Replit , Atlassian and more. Discussions at the conference… 31 r/LocalLLaMA community 20d ago Mac Studio M5 Ultra 96GB vs M5 Max 128GB for local LLMs? I'm about to buy a Mac Studio mainly for running LLMs locally and I'm stuck between two configs: M5 Ultra (30/64) with 96GB : 1.2 TB/s bandwidth, roughly 1.7x faster generation and much faster prefill M5 Max (40-core GPU) with 128GB : 614 GB/s, but 32GB more memory and a bit… 15 r/LocalLLaMA community 20d ago MiniMax M3.1 (Space Bunny Alpha) thinks in caveman mode The CoT of MinimMax M3.1, currently available in openrouter and opencode under the guise of "Space Bunny Alpha", has the familiar look of caveman mode in order to save tokens. This has no impact on the final output. (note: in the first screenshot, pi-caveman is set to off; in… 30 llama.cpp releases dev-tools 20d ago b11158 vulkan: tune KHR cooperative matrix support for Adreno GPUs ( #29328 ) Enable coopmat support for Vulkan backend Fixed the mul_mat_s Removed the debug statement Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49831613 macOS/iOS: macOS… 35 llama.cpp releases dev-tools 20d ago b11157 cuda : add conv3d with implicit GEMM ( #29137 ) cuda : add conv3d with implicit GEMM cuda : refine conv3d implicit GEMM and handle empty kernels Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49789544 macOS/iOS: macOS Apple Silicon… 21 r/LocalLLaMA community 20d ago Folks, have you purchased the Mac M5 Ultra with 256GB yet? We need serious benchmarks, because we only get YouTube clowns influencers results Ok, so the Mac M5 Ultra (256GB) hit the market, but the only publicly available benchmarks material are flashy YouTube "clown influencers" videos. We need serious numbers to evaluate whether Apple’s silicon can actually compete with Nvidia’s current GPU‑centric workflows or not.… 11 llama.cpp releases dev-tools 20d ago b11154 test-save-load-state : print a per-model results table in --models mode ( #29316 ) test-save-load-state : print a per-model results table in --models mode in --models mode the output was very heavy: every model printed its token dumps, per-test headers and PASS lines. instead,… 35 r/LocalLLaMA community 20d ago My foray into local ai. Two BC-250 ex mining apus running Qwen3.6-35B-A3B Q4_K_M at 60 tok/s with 64k context These boards cost me $115 each and I have them connected using llama.cpp with Vulkan and RPC on Bazzite. The boards have roughly 27GB of combined GPU memory and communicate over 1gb Ethernet. For around $300 including psu I’m loving the performance. I have a few more and want to… 22 arXiv — Machine Learning research 20d ago EBRL: Asynchronous Embodied RL by Multi-Grained Resource Management arXiv:2609.27547v1 Announce Type: new Abstract: Embodied reinforcement learning (RL) improves model capabilities with a pipeline of environment simulation, action generation, and model updates. These stages show heterogeneous CPU and GPU demands, making efficient resource… 7 arXiv — Machine Learning research 20d ago NS-ATTENTION: Newton-Schulz Transformations of Attention Outputs in Vision Transformers arXiv:2609.27735v1 Announce Type: new Abstract: Newton-Schulz (NS) iteration has recently been used in the Muon optimizer to transform update matrices during the training of large language models. Motivated by its spectral effect, we investigate applying NS directly to… 29 arXiv — Machine Learning research 20d ago Binary Quantized Neural Network Training Is W[1]-Hard Parameterized by Input and Output Dimensions arXiv:2609.27932v1 Announce Type: new Abstract: Ganian et al. (ICLR 2026) proved that quantized neural network training is fixed-parameter tractable when parameterized jointly by architecture treewidth, input dimension $\alpha$, and output dimension $\omega$, and left open… 11 arXiv — NLP / Computation & Language research 20d ago COMED: The Missing Middle Between Routing and Collaboration in Multi-LLM Inference arXiv:2609.26913v1 Announce Type: new Abstract: No single Large Language Model (LLM) is uniformly reliable across queries, motivating multi-model inference systems that either route among models or combine their outputs. However, routing stops after selecting an initial model,… 7 arXiv — NLP / Computation & Language research 20d ago Brain-to-Language Decoding: Tasks, Signals, Methods, Evaluation, Practical Use and Beyond arXiv:2609.27650v1 Announce Type: new Abstract: Brain-to-language decoding translates neural activity associated with language production, internal speech and perception into linguistic or expressive outputs. It offers a route to restoring communication after speech loss and a… 25 arXiv — NLP / Computation & Language research 20d ago Predicting Quantization Price for Selecting PTQ Configurations Before Deployment arXiv:2609.28270v1 Announce Type: new Abstract: Weight-space post-training quantization (PTQ) must choose finite formats, granularities, quantizer families, transformations, and bits before the completed quantized model reveals its output-distribution drift. Existing PTQ methods… 20 arXiv — NLP / Computation & Language research 20d ago Text Scores Can Miss Waveform Use: A Qwen2-Audio Quantization Case Study arXiv:2609.26823v1 Announce Type: cross Abstract: Post-training quantization of speech language models is often summarized with text-output scores and nominal bit widths. Those numbers alone do not establish behavior that depends on information missing from a transcript, or… 38 Ollama releases dev-tools 21d ago v0.34.4-rc1: mlxrunner: Update XGrammar to 0.2.7 for structured outputs We pick up schema fixes for typed dictionary values and short arrays. 12 Ollama releases dev-tools 21d ago v0.34.4: mlxrunner: Update XGrammar to 0.2.7 for structured outputs We pick up schema fixes for typed dictionary values and short arrays. 18 NVIDIA Developer Blog official-blog 21d ago Validate GPU Cluster Readiness Before AI Workloads Land A GPU cluster can pass every health check and still fail to run an AI workload. Even when every GPU, network link, and pod reports healthy, a 512-GPU training... 28 Hugging Face official-blog 21d ago How to Use NVIDIA Warp and MjWarp to Accelerate Robotics Simulation and Learning Workflows Back to Articles a]:hidden"> How to Use NVIDIA Warp and MjWarp to Accelerate Robotics Simulation and Learning Workflows Enterprise + Article Published September 23, 2026 Upvote - Johnny Nuñez Cano johnnynv nvidia Asier Arranz asiernvidia nvidia Rishabh Chadha rchadha-nv nvidia… 9 r/LocalLLaMA community 21d ago M5U base 96GB inference numbers for Q3.8FN after 112M tokens TLDR; Base M5 Ultra 96 GB ran Q3.8 FN aggregate 3.2k PP and ~170 TG in 4 concurrency Alert: Numbers and custom server details at end are AI assisted So the good news is that I got the base model on launch day with only 64 core GPU. All benchmarks are for current maxed out model,… 34 r/LocalLLaMA community 21d ago Jev isn't new tech. Its marketing targets people who think AI started with LLMs. I keep seeing Jev presented as some new class of decision model, but most of what’s being advertised is just normal classifier behavior with modern zero-shot capabilities. It outputs probabilities over constrained choices, doesn’t generate autoregressively, can’t output an… 10 llama.cpp releases dev-tools 21d ago b11140 CUDA: enable sparse-fa for dsv4 prefill (again) ( #29298 ) CUDA: enable sparse-fa for dsv4 prefill (again) CUDA: unroll the query loop of the sparse mask scan The query loop of flash_attn_mask_to_sparse_indices has a runtime trip count, which keeps the unrolled scan over the… 35 r/LocalLLaMA community 21d ago Streaming Nemotron 3 Diarization I’ve been playing with Nemotron 3 Diarization , and it fills a gap I’ve had with local voice agents: keeping track of who is speaking. It’s a diarization model, so it gives you speaker labels rather than transcriptions or people’s names. It can stream its output and track up to… 4 r/LocalLLaMA community 21d ago Perhaps the highest quality mainline quants of Qwen3.8 27B? I am proud to release these quants of Qwen 3.8 27B. They beat the excellent ISTA and Unsloth quants byte-for-byte on three corpora. Both KLD and top 1% were tested 3x. It took a week of continuous GPU and CPU time to generate these, all done on a single Strix Halo.… 4 llama.cpp releases dev-tools 21d ago b11132 model : support Gemma4 DSpark draft backbone ( #29226 ) dspark: add Gemma 4 draft support Add GGUF conversion and runtime support for full-attention and SWA Gemma 4 DSpark drafts, including tied output weights and boolean backbone metadata. Assisted-by: Codex dflash: infer Gemma… 8 Hugging Face official-blog 21d ago **Know Who Spoke When: Build Real-Time, Multi-Speaker AI with NVIDIA Nemotron 3 Diarization** Back to Articles a]:hidden"> Know Who Spoke When: Build Real-Time, Multi-Speaker AI with NVIDIA Nemotron 3 Diarization Enterprise + Article Published September 23, 2026 Upvote 2 Francesco fciannella nvidia Ivan Medennikov imedennikov nvidia Taejin Park taejinp nvidia Adi-… 12 llama.cpp releases dev-tools 21d ago b11124 cuda: top-k MoE should always fire ( #28432 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49495074 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework… 23 arXiv — NLP / Computation & Language research 21d ago Slow Decay and Silenced Expression: Iterated Subliminal Trait Transfer in Language-Model Lineages arXiv:2609.25721v1 Announce Type: cross Abstract: Language models are increasingly trained on the outputs of other models, forming chains that we call lineages, in which a trait present in one generation can pass to the next. Prior work on subliminal learning has shown that a… 11 arXiv — NLP / Computation & Language research 21d ago Training a Language Model End-to-End in Rust: An Experience Report arXiv:2609.25008v1 Announce Type: new Abstract: I pretrained a language model end-to-end in Rust - alone, with no team, no PyTorch, and no Python in the training path - for $164 in rented GPU time. I report that as an achievement, not a recommendation: the more useful… 17 arXiv — NLP / Computation & Language research 21d ago Matryoshka attribution: Learning to attribute language model outputs to representations and weights arXiv:2609.25518v1 Announce Type: new Abstract: Attributing language model outputs to their internal computations is an open problem in interpretability. Existing methods, which use causal interventions, gradients, or learnable masks, either are infeasibly expensive or struggle… 36 arXiv — NLP / Computation & Language research 21d ago Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference arXiv:2609.25537v1 Announce Type: new Abstract: Large language model (LLM) inference is constrained by the quadratic scaling of self-attention and the linear scaling of the KV cache, increasing latency, energy consumption, and GPU memory demand as context length scales. Existing… 24 arXiv — NLP / Computation & Language research 21d ago Modality-Gated Deep Adapters: Adding a Modality to a Frozen Embedding Model with Exact Preservation arXiv:2609.26182v1 Announce Type: new Abstract: Multimodal embedding models are deployed at scale: retrieval indices, benchmark results, and behavioral audits all depend on the base model's exact outputs. Extending such a model to a new modality with existing parameter-efficient… 29 arXiv — NLP / Computation & Language research 21d ago Layout-Guided Masking for GROBID: Lightweight Structural Gains in Large-Scale Scientific PDF Ingestion arXiv:2609.26381v1 Announce Type: new Abstract: Transforming scholarly PDFs into machine-readable fulltext remains a bottleneck for large-scale information systems. Recent vision-based parsers improve accuracy, but need GPUs and may introduce noise into the extracted text.… 34 arXiv — NLP / Computation & Language research 21d ago Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decoding arXiv:2609.26638v1 Announce Type: new Abstract: Autoregressive OCR vision-language models accurately convert document images into text and structured markup, but require one sequential decoding step per output token, limiting inference speed. Unlike open-ended text generation,… 5 arXiv — NLP / Computation & Language research 21d ago Efficient Iterative Retrieval with Heterogeneous Batching arXiv:2609.25405v1 Announce Type: cross Abstract: Modern information retrieval increasingly employs both embedding and generative models to handle complex queries. However, current serving systems suffer from low throughput and poor GPU utilization because they execute these… 23 LangChain releases dev-tools 22d ago langchain-anthropic==1.7.3 Changes since langchain-anthropic==1.7.2 release(anthropic): 1.7.3 ( #40773 ) chore(model-profiles): refresh anthropic model profile data ( #40772 ) fix(anthropic): auto-route with_structured_output to method="json_schema" for fable and opus 5.5 ( #40766 ) chore(anthropic):… 38 r/LocalLLaMA community 22d ago My local llm when I tell it to do any changes to my vLLM service better make no mistakes I've been running Qwen 3.8 Flash Next and it's a great driver for Hermes and Pi. I told it to add CUDA_DISABLE_PERF_BOOST=1 to reduce my server's idle power draw   submitted by   /u/ZaltyDog [link]   [comments] 38 The Information — AI news-outlet 22d ago Wall Street’s GPU Futures Push Stalls at the CFTC The launch of futures for rental prices on Nvidia GPUs, which are seen as important to make AI compute an investable asset class, is hitting a snag. CME Group had hoped to launch contracts as soon as early October, but it won’t have regulatory approval by then. The Commodity… 15 llama.cpp releases dev-tools 22d ago b11112 server: support input_image in function_call_output ( #20663 ) ( #22575 ) server: support input_image in function_call_output ( #20663 ) server: fix if statement spacing server: avoid repeated type lookup Website: https://llama.app Attestations:… 34 r/LocalLLaMA community 22d ago MiMo-V2.6-Flash on vLLM: fixes for "empty responses" with thinking + tools, and a hidden 2,048-token output cap Some people here say MiMo-V2.6 is bad with tools and are going back to GLM-5.3-Flash. I spent today running MiMo-V2.6-Flash-RL as the backend for an agent harness, on 2× DGX Spark with vLLM, using the tonyd2wild recipe. Most of the "tool problems" I hit turned out to be serving… 8 Page 8 of 10 · 500 articles ← Newer Older →
