News / #gpu Tag Gpu 500 articles archived under #gpu · RSS Sign in to follow NVIDIA Developer Blog official-blog 13d ago Fine-Tuning NVIDIA Nemotron for Saudi Arabic Dialects, with a Path to Other Languages Automatic speech recognition must handle how people actually speak, not only the languages and styles that dominate pretraining data. Regional dialects and... 35 arXiv — NLP / Computation & Language research 13d ago Concept Subspaces Compute Beyond the Logit Lens: A Weights-Only Test for Locating Representations Upstream of Readout arXiv:2609.39263v1 Announce Type: new Abstract: A concept subspace's effect on model behavior does not establish how it relates to the output readout. We introduce a two-sided geometric diagnostic that measures an extracted subspace's overlap with the dominant right-singular… 11 arXiv — NLP / Computation & Language research 13d ago Making Grid Beam Search Less Greedy arXiv:2609.39368v1 Announce Type: new Abstract: A common formalism for constraining the output of autoregressive text generation models involves lexical constraints, words or phrases which are required to occur in the generated text. DFA-constrained beam search and grid beam… 17 arXiv — NLP / Computation & Language research 13d ago Stress-Testing LLM Lie Detectors: Role-Play Failures and Spurious Correlations arXiv:2609.39807v1 Announce Type: new Abstract: Lie detection probes aim to predict from a language model's internal states whether its output is truthful or dishonest. However, role-play complicates what "truth" means for an LLM: language models can adopt a wide range of… 36 arXiv — NLP / Computation & Language research 13d ago Cognitive Enhancement: Rethinking the Necessity of Role-Playing for Large Language Models arXiv:2609.39853v1 Announce Type: new Abstract: Role-playing prompting has become a popular yet simple technique for improving LLM reasoning and output quality. However, whether it consistently boosts performance across diverse domains remains unclear, as systematic validation… 31 llama.cpp releases dev-tools 13d ago b11312 webgpu: fix SSM_SCAN binding aliasing ( #29750 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51685637 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework… 24 llama.cpp releases dev-tools 13d ago b11310 cuda: guard the iq4_nl dequantize row kernel against short rows ( #29683 ) dequantize_block_iq4_nl writes QK_K values per block, but a row can be shorter than that (an IQ4_NL row is only guaranteed to be a multiple of QK4_NL). Threads whose 32-value sub-block starts at or past k… 4 r/LocalLLaMA community 14d ago tp=6 can work on vLLM, with padding vLLM's tensor parallel requires that several of the model architecture numbers be evenly divisible by the number of GPUs selected for tensor parallel. This usually means that you can only use a number of GPUs that is a power of two (2, 4, 8, etc.). I have six GPUs, and I want to… 8 llama.cpp releases dev-tools 14d ago b11307 llama : preserve original batch order for speculative decoding layer inputs ( #29019 ) llama: preserve original batch order for layer inputs Assisted-by: Codex tests: cover layer-input order across KV layouts Assisted-by: Codex tests: exercise layer-input ordering on CUDA… 23 r/LocalLLaMA community 14d ago Least to most expensive (Somewhat modern) GPU's with 32gb of vram (Under $1600) Based on ebay listings I was researching prices on ebay and fed claude a bunch of images of listings. I had it make a chart and thought it would be useful to share.   submitted by   /u/Rombodawg [link]   [comments] 16 r/LocalLLaMA community 14d ago Open source inference engine (like LM Studio or Unsloth Desktop) that optimizes itself for your exact hardware. Compiles and tunes its kernels on your device, so open models run up to 2x faster than llama.cpp. Works on Apple Silicon, NVIDIA, AMD or nothing but a CPU.   submitted by   /u/paranoidray [link]   [comments] 34 NVIDIA Developer Blog official-blog 14d ago Deploying an HSTU Generative Recommender with NVIDIA Dynamo-Triton Generative recommender (GR) systems are emerging as a powerful new direction for large-scale personalization. Instead of treating recommendation as a set of... 24 NVIDIA Developer Blog official-blog 14d ago Expanding AI Storage Access with NVIDIA cuObject and the NVIDIA SCADA Server SDK AI infrastructure engineers, storage developers, and cloud service providers need fast and secure access to high-capacity file and object storage to support AI... 10 llama.cpp releases dev-tools 14d ago b11295 ci : fix Models Backend Check by shortening the hrm_text fixture ( #29744 ) The fixture recycles its two blocks over 8 cache slots, so the fp16 error builds up past the 1e-4 NMSE bound on the Vulkan T4 and WebGPU jobs of Models Backend. Two l-cycles keep every branch of the… 9 llama.cpp releases dev-tools 14d ago b11293 ggml : add BF16 unary, GLU, binary and scale ops (CPU, CUDA) ( #29675 ) ggml : add BF16 unary, GLU, binary and scale ops (CPU, CUDA) ggml-cpu : use per-op _bf16 functions for BF16 unary and GLU ops Assisted-by: Claude Opus 5.5 CUDA: use ggml_cuda_cast in binbcast and unary… 38 r/LocalLLaMA community 14d ago How smart is the IQ3 family of Qwen 3.8 Flash Next for coding tasks? I've been closely following the rise of the Strata inference engine and as someone with 28GB VRAM and 32GB RAM I'm itching to buy 32GB more RAM just to use Flash Next. But of course before I make such a financial commitment as a member of the GPU-poor class like myself, first I… 14 r/LocalLLaMA community 14d ago We just open-sourced the world's fastest WebGPU kernels for local AI on Hugging Face The collection includes kernels for more than 200 common ML operations, all of which can run entirely locally in your browser on WebGPU. We're also working to upstream these optimizations to Transformers.js, ONNX Runtime Web, LiteRT.js, and more! Kernels:… 14 NVIDIA Developer Blog official-blog 14d ago Tracing Agent Harness Behavior with NVIDIA NeMo Relay An agent can finish a task and still take an inefficient path. A failed search can trigger another search. A truncated file read can lead to a command fetching... 22 llama.cpp releases dev-tools 14d ago b11282 musa : define CUDA_ARCH for device passes ( #29508 ) The MUSA vendor header never defined CUDA_ARCH , so every architecture test in the shared ggml-cuda sources evaluated to 0. Kernel bodies gated on the architecture therefore compiled to nothing, for example the q8_0 -> f16… 20 r/LocalLLaMA community 14d ago Oído: speech recognition that beats Whisper-tiny, running on a $5 microcontroller (open source) I'm part of the Lokutor team that built this. Model: NVIDIA Conformer-CTC Small (13M params, int8). It runs on an ESP32-S3 with 8 MB PSRAM, no GPU or NPU. LibriSpeech WER is 3.7 / 8.2, versus 6.3 / 15.9 for Whisper tiny.en on a laptop. Under real noise (DEMAND: car, kitchen,… 16 The Information — AI news-outlet 14d ago Exclusive: GPU Cloud Provider GMI Raises $668 Million From Nvidia and Others GMI Cloud, a five-year-old provider of Nvidia chip servers to enterprises, said Wednesday it raised $668 million in equity and debt financing. The new fundraise indicates Nvidia is continuing to help smaller, upstart cloud providers, especially those that plan to boost Nvidia in… 19 Hacker News — AI on Front Page community 14d ago OpenDLSS: A Vulkan Reimplementation of Nvidia's DLSS 5 Neural Rendering Network Article URL: https://github.com/maanHimself/OpenDLSS-NR Comments URL: https://news.ycombinator.com/item?id=49906100 Points: 203 # Comments: 98 12 r/MachineLearning community 14d ago LessThink-Qwen3-4B: the same model, with far less thinking [P] I post-trained Qwen3-4B to spend 44% fewer tokens on reasoning, keeping its knowledge and answer style. The whole pipeline ran on one GPU. folks, you can check it out on : https://5ivatej.com/lessthink/   submitted by   /u/stey1r [link]   [comments] 30 arXiv — Machine Learning research 14d ago The Decision Value of Perception Compute arXiv:2609.35910v1 Announce Type: new Abstract: Adaptive perception spends extra computation on inputs where perception is expected to improve. When perception feeds a downstream decision system, a better perception output need not produce a better decision. We define the… 18 arXiv — Machine Learning research 14d ago GEM-KMeans: Memory-Efficient and Accurate Clustering on Massive Scale with GPU Optimization arXiv:2609.36074v1 Announce Type: new Abstract: Memory-efficient scaling on clustering problems without sacrificing statistical accuracy is of central interest for large-scale data analysis and machine learning problems. Nonnegative low-rank (NLR) matrix factorization for… 17 arXiv — Machine Learning research 14d ago Data Unlearning via Inverse Distillation arXiv:2609.36099v1 Announce Type: new Abstract: Multi-step matching models, including flow and diffusion models, produce high-quality outputs but incur substantial inference costs and may reproduce unwanted components of their training datasets. We introduce Inverse Distillation… 12 arXiv — Machine Learning research 14d ago Optimal Multi-Reward Reinforcement Learning arXiv:2609.36486v1 Announce Type: new Abstract: We study an unknown-transition finite-horizon Markov decision process (MDP) with a finite collection of known reward functions $\{r^1, r^2, \ldots, r^M\}$. The goal is to output an $\epsilon$-optimal policy for every reward using… 35 arXiv — Machine Learning research 14d ago LLMs Learn to Evade Latent Monitors from Prior Feedback Alone arXiv:2609.36490v1 Announce Type: new Abstract: Latent space monitors aim to detect undesired behaviors in LLM agents by inspecting an agent's internal activations rather than its outputs. However, interactive monitoring creates a feedback channel where each verdict the monitor… 9 arXiv — NLP / Computation & Language research 14d ago Environment Steering: Using Data Flow Control to Improve Agent Utility and Safety arXiv:2609.35807v1 Announce Type: new Abstract: LLM agents can make unsafe tool calls even when instructed to behave safely. Existing defenses constrain agents before execution, modify tool inputs/outputs, or rely on LLM judges; these approaches may depend on model behavior or… 36 arXiv — NLP / Computation & Language research 14d ago Language Models Are "Insecure" Reporters arXiv:2609.36139v1 Announce Type: new Abstract: As large language models are deployed in increasingly autonomous long-horizon tasks, manually auditing and verifying the actions, artifacts, and outputs of models becomes more difficult. Users instead come to rely on LLM-generated… 32 arXiv — NLP / Computation & Language research 14d ago Distilling What Matters: Confidence-Aware Selective Distillation for Large Language Models arXiv:2609.36734v1 Announce Type: new Abstract: Knowledge Distillation (KD) trains a smaller-capacity student model to imitate a larger-capacity teacher model by matching output distributions, implicitly assuming the teacher to be a reliable oracle. In large language models… 15 arXiv — NLP / Computation & Language research 14d ago Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation arXiv:2609.36931v1 Announce Type: new Abstract: Reproducibility is essential for scientific research, yet prior work shows that LLM outputs vary with hardware and batching. We identify an overlooked factor: the hidden injection of the current date into system prompts, which… 8 r/LocalLLaMA community 15d ago Sherry's 3:4 ternary format (1.375 bits per weight) running on WebGPU: a 1.6 MB model that plays Connect Four as well as its 7.8 MB int8 version Not an LLM, but the ternary findings should carry over, and we hadn't seen Sherry-style 3:4 weights run in a browser before. Disclosure: this is our work at Precisit, everything is MIT. What it is A 7.4M-parameter one-pass scorer (the jevlike family): the board goes in, one… 19 r/LocalLLaMA community 15d ago AMD boosting AI/LLM performance for Radeon iGPUs as much as 18~23% with Linux 7.4   submitted by   /u/Fcking_Chuck [link]   [comments] 33 Ars Technica — AI news-outlet 15d ago AMD acquires World Labs AI startup, upping the ante against Nvidia The deal, which is expected to close by year's end, is worth $8.2 billion. 9 The Information — AI news-outlet 15d ago Google Joining Industry Group Working to Help AI Better Understand Data Google is in the process of joining Apache Ossie, a software industry group with members such as Snowflake and Nvidia that is trying to make it easier for AI tools to access data from applications and databases, according to a company spokesperson. The search and cloud provider… 18 NVIDIA Developer Blog official-blog 15d ago AI Native by Design: Lessons Learned from Building NVIDIA TensorRT Model Connect Parallel work, model-family isolation, reversible changes, and GPU-backed validation shaped an open source project designed around coding agents NVIDIA TensorRT... 16 NVIDIA Developer Blog official-blog 15d ago Lower the Cost of Building and Running Visual AI Agents with NVIDIA VSS Blueprint 3.3 Vision-language models have made it possible to build visual AI agents that understand video at production scale. The harder problem is turning that capability... 5 TechCrunch — AI news-outlet 15d ago Here’s why OpenAI is absent from Nvidia’s industry-wide effort to end rogue AI agents OpenAI isn't a public supporter of Nvidia's Open Agent Safety Platform, but it is privately working with Nvidia, TechCrunch has learned. 38 Hugging Face official-blog 15d ago NVIDIA Kumo Tabular Sets a New Accuracy-Efficiency Frontier for Tabular Prediction Back to Articles a]:hidden"> NVIDIA Kumo Tabular Sets a New Accuracy-Efficiency Frontier for Tabular Prediction Enterprise + Article Published September 29, 2026 Upvote 2 Jingang Qu jingangqu nvidia Valter Hudovernik valterh-nv nvidia Martin Jurkovic martinjurkovic nvidia Cedric… 14 The Information — AI news-outlet 15d ago Anthropic Discloses up to $84.5 Billion in SpaceX Compute Agreements Anthropic has agreements to pay SpaceX up to $84.5 billion to use the rocket and AI company’s Nvidia-based computing capacity through 2029, according to a confidential initial public offering prospectus reported by Reuters on Tuesday. Those agreements can mostly be cancelled… 36 r/LocalLLaMA community 15d ago Qwen 3.8 27B Q4 with 100K context on a 16 GB RX 7800 XT guide I'm running Qwen 3.8 27B Q4 XS with ~30 t/s decode (no MTP) at 100k context with q8/q5 KV cache on a 16GB AMD GPU. I wanted to share my setup since many believe Qwen 3.8 27B to be infeasible on 16GB VRAM. Build llama.cpp with Vulkan: cmake -B build -DGGML_VULKAN=ON && cmake… 4 OpenAI official-blog 15d ago Introducing GPT-6.1 Sol Meet GPT-6.1 Sol: near-Astra intelligence for coding, computer use, and professional work at one-fifth of Astra’s standard API input and output token prices. 6 r/MachineLearning community 15d ago Why not just have one less feature before softmax? [D] Softmax has N inputs and N outputs but it's output only has N-1 degrees of freedom because of the condition that the sum of outputs must be equal to one. Based on this we can figure out that actually we can make due with only N-1 inputs by making an assumption that logits before… 4 r/LocalLLaMA community 15d ago Do you need some extra memory on your DGX Spark? ​ I created this repo to help the DGX Spark users that have a spare 10-24 GB GPU at home to squeeze some extra memory out of a single Spark or a Sparks cluster. It moves the spec-decode draft model off your Sparks onto that GPU: the freed GB of memory can be used for… 28 arXiv — Machine Learning research 15d ago Entropy Regularization: A Free Correction to Cross-Entropy for Verified Demonstrations arXiv:2609.30572v1 Announce Type: new Abstract: Large language models are often post-trained on expert demonstrations using cross-entropy (CE), even when the downstream objective is not to imitate the demonstrated solution but to produce any output accepted by a verifier. This… 12 arXiv — Machine Learning research 15d ago Differentiable RNA Secondary Structure Extraction for Deep Learning arXiv:2609.30752v1 Announce Type: new Abstract: Many deep learning approaches to RNA secondary structure prediction have recently been proposed. They typically output a weight matrix $W$ where $W_{ij}$ is an arbitrary weight for base $i$ pairing with base $j$. Converting this… 34 arXiv — Machine Learning research 15d ago From Shortcut Learning to Discrete Neural Insertion Sort arXiv:2609.31114v1 Announce Type: new Abstract: Neural algorithmic reasoning aims to train neural networks to follow known algorithms and generalize beyond the input sizes seen during training. However, correct final outputs and intermediate supervision do not necessarily show… 24 arXiv — Machine Learning research 15d ago Softmax Reparameterization for Output-Head Quantization arXiv:2609.31291v2 Announce Type: new Abstract: Large vocabularies make output heads a substantial inference cost in small language models. We introduce softmax reparameterization, a post-training method that searches over functionally equivalent output heads before… 32 vLLM releases dev-tools 15d ago v0.31.0rc1: [CI/Build] Skip the snapshot runtime on CUDA 12.x images (#59118) Signed-off-by: khluu khluu000@gmail.com Co-authored-by: Claude Opus 5.5 noreply@anthropic.com (cherry picked from commit aedaba8 ) 33 Page 6 of 10 · 500 articles ← Newer Older →
