News / #gpu Tag Gpu 500 articles archived under #gpu · RSS Sign in to follow llama.cpp releases dev-tools 6d ago b11498 CUDA: fix norm family kernels when ne[2]/ne[3] exceed grid dim limits ( #28175 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53919465 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED… 21 Ars Technica — AI news-outlet 6d ago Nvidia's big bet on physical AI aims for safer robotaxis, humanoid robots Full-stack safety solution for physical AI is being used by robotics companies. 37 r/LocalLLaMA community 6d ago Best current R9700 inference engine? There are way too many forks to keep track of, so I've gotten lost. As far as I can tell, Radiance VLLM is best for models that fit in GPUs while some form of llama.cpp is probably best for MOE RAM-spill? My specific current goal is to run GLM5.3-Flash over 6 R9700s + system RAM… 32 llama.cpp releases dev-tools 6d ago b11502 cuda : support arbitrary striding for unary ops on f16, f32, and bf16… 23 llama.cpp releases dev-tools 6d ago b11499 cuda : Use byte strides for roll to allow non-contiguous ROLL operati… 31 r/MachineLearning community 6d ago Nvidia’s erroneous paper accepted as ICML’s spotlight [D] Here is the story: Nvidia has published this work (with source code available) called dreamDojo which is a world model for robotics based of their prior work Cosmos 2.5 which is cited about 100 times and got ICML’s spotlight. Authors are very well known and respected in the… 8 arXiv — Machine Learning research 6d ago The Symbol of the Surrogate: Measuring Numerical Provenance in Neural PDE Solvers arXiv:2610.09255v1 Announce Type: new Abstract: Neural PDE surrogates are trained on numerical solver outputs that contain both physical evolution and solver-specific discretization errors. Because surrogates are also evaluated against held-out trajectories from the same solver,… 38 arXiv — Machine Learning research 6d ago The Identifiability and Observability of Deep Normalized Attention arXiv:2610.09620v1 Announce Type: new Abstract: We study which parameters of deep, unmasked, single-head attention are determined by its input--output function. For known positive nonconstant real-analytic normalizers, the function generically determines the effective scores and… 4 arXiv — NLP / Computation & Language research 6d ago Expert Coupling in MoE Pretraining: Reducing All-to-All Overhead with Correlated Placement and Token Shuffling arXiv:2610.09372v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) layers replace the feed-forward block of a Transformer with E expert networks, and each token is routed to k of these experts. Under expert parallelism (EP) the experts are distributed across GPUs, and… 16 arXiv — NLP / Computation & Language research 6d ago A Deafening Silence: Catastrophic Forgetting Lives in the Output Embeddings of Tokens the Data Never Speaks arXiv:2610.09835v1 Announce Type: new Abstract: Continual pre-training and fine-tuning in Large Language Models (LLMs) inevitably induce catastrophic forgetting, typically mitigated by replay using often-inaccessible original data. In this data-free regime, we analyze where… 35 arXiv — NLP / Computation & Language research 6d ago Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution arXiv:2610.09240v1 Announce Type: cross Abstract: Modern web agents built on large vision-language models process webpages, select relevant UI elements, and translate model outputs into browser actions. Existing visual red-teaming approaches use adversarial visual content to… 14 r/MachineLearning community 6d ago Instead of another GPU terminal renderer, I trained a 1.26M-param model to turn TUIs (htop, vim, emacs…) into real UI components [R] Write-up + demo: https://drksci.com/labs-phosphene So this started as a bit of a gripe. Modern terminal renderers are seriously impressive and seriously complicated. GPU glyph atlases, texture caches, custom shaders, HarfBuzz shaping, ligatures, damage tracking, grid diffing,… 20 llama.cpp releases dev-tools 6d ago b11482 cuda: FWHT kernels for block widths above 512 ( #29100 ) The CUDA FWHT covers widths 64 to 512. It runs one row per warp and keeps N/32 values per lane, so wider blocks need more registers per lane than that layout allows. fwht_cuda_block runs one row per thread block with 256… 20 llama.cpp releases dev-tools 6d ago b11488: model : add LiquidAI/d1-omni-600M decision model (#30114) model : add LiquidAI/d1-omni-600M decision model Assisted-by: Claude Opus 5.5 mtmd : keep conformer GLU sigmoid on CUDA Assisted-by: Claude Opus 5.5 server : take d1omni audio through images and input_audio, scope memory-less lfm2 to non-causal Assisted-by: Claude Opus 5.5… 11 r/LocalLLaMA community 7d ago Surface RTX Spark Dev Box: The Dev Box Built For Developers $5995 - Ships in November. N1X brand of GB10 Chip. Says it will have WSL out the door which I presume will be Ubuntu + Cuda beneath in addition to all the CoPilot/GitHub native stuff for Windows. Hopefully they have supply to saturate the market and put in pricing pressure.… 17 Simon Willison community 7d ago Introducing Claude Haiku 5.5 As previously promised , here's Anthropic's new fast, low cost model: Introducing Claude Haiku 5.5 . The previous Haiku, 4.5, was very much showing its age. It came out almost a year ago , and was priced at $1/million input and $5/million output - relatively expensive even back… 9 TechCrunch — AI news-outlet 7d ago Microsoft releases new Nvidia-chip AI PCs with revamped Windows 11 Microsoft revealed the specs and price for its Surface Laptop Ultra, AI PCs that run on Nvidia chips that are designed to run AI models and agents. 13 r/LocalLLaMA community 7d ago llama : add a GPU cache for MoE experts kept in host memory by am17an · Pull Request #29887 · ggml-org/llama.cpp Potentially big speedup for MoE models that don’t fully fit in VRAM. Are you GPU Poor? Show your speedups ;)   submitted by   /u/jacek2023 [link]   [comments] 37 llama.cpp releases dev-tools 7d ago b11480: llama : add a GPU cache for MoE experts kept in host memory (#29887) llama : add a GPU cache for MoE experts kept in host memory Assisted-by: Claude use llama_moe_cache_ptr 29 r/LocalLLaMA community 7d ago omni-d1 600M by Liquid AI running in the browser using WebGPU Liquid just dropped d1-omni-600M and it's a decision model! I ported it to runntime, the WebGPU inference library I'm currently working on. It's plain TypeScript on top of TypeGPU with no WASM and virtually no export step. The model is written directly from our core ops (matmul,… 34 llama.cpp releases dev-tools 7d ago b11479: cuda: update uncoalesced memory reads in pool2d (#29425) cuda: update uncoalesced memory reads in pool2d cuda: Added perf and restrict in pool2d Update POOL2D_WARP_KERNEL_MIN_WINDOW macro Co-authored-by: Georgi Gerganov ggerganov@gmail.com cuda: Fix compiler bugs Co-authored-by: Georgi Gerganov ggerganov@gmail.com 26 NVIDIA Developer Blog official-blog 7d ago Validate AI Factory Changes with Digital Twins and AI Agents AI factories are some of the most complex operations in the world, combining GPUs, CPUs, switches, DPUs, and SuperNICs alongside schedulers, orchestration... 23 NVIDIA Developer Blog official-blog 7d ago Scaling Decision Optimization to 100 Million Variables and Beyond with mPDLP in NVIDIA cuOpt Supply chain problems are expanding across more SKUs, lanes, and constraints than ever before, while energy grids are balancing more distributed sources in real... 30 The Information — AI news-outlet 7d ago AI Agenda Live: From Token Maxing to Value maxing: Getting More From Every Unit of Compute In AI circles, conversations about compute shortages are as common as talk of the weather. As agentic workflows threaten to multiply token consumption further, it’s little wonder that talk has shifted from securing GPUs to optimizing them. At a recent panel, The Information’s… 15 NVIDIA Developer Blog official-blog 7d ago Faster Scientific Image Analysis with NVIDIA cuPhoton Observatories and telescopes, lasers and X-ray light sources, and other high-throughput instruments generate image data faster than CPU-bound pipelines can... 18 TechCrunch — AI news-outlet 7d ago Get hands-on: The full lineup of interactive roundtables at TechCrunch Disrupt 2026 From Nvidia and Chime to Obvious Ventures and Anthropic, explore the entire roundtable agenda at TechCrunch Disrupt 2026. Register now to save up to $100 on your pass and get a second pass at 50% off. 37 r/LocalLLaMA community 7d ago GPU Upgrade advice Upgrade Question If we say the "budget max" is $1700-1800, and this could include upgrading to a Taichi motherboard: With a new motherboard , no need to bifurcate 1. Would you add a second 5060Ti (16GB)? Someone is selling one for around $500 locally; 2. Buy a used 7900 XTX… 35 The Information — AI news-outlet 7d ago Where Nvidia’s $100 Billion Dealmaking Juggernaut Will Go Next This July, Nvidia caught wind of a looming takeover of OpenRouter, a three-year-old startup that had emerged as a popular marketplace for AI models, including many cheaper open-source ones. Databricks and Stripe had already been talking to the startup about a potential sale.… 35 Hugging Face official-blog 7d ago One Model Family, Two Gold-Level Results: Fine-Tuning Nemotron for IOI and IMO Back to Articles a]:hidden"> One Model Family, Two Gold-Level Results: Fine-Tuning Nemotron for IOI and IMO Enterprise + Article Published October 7, 2026 Upvote 2 Aleksander aficek nvidia Igor Gitman igitman nvidia Sean Narenthiran SeanNaren nvidia Mehrzad Samadi mehrzads… 25 The Information — AI news-outlet 7d ago Former Google, Nvidia Execs Launch Company to Ease GPU Crunch Anjney Midha, a former general partner at Andreessen Horowitz, and former executives at Google, Apple and Nvidia have launched a company that aims to make it easier and more affordable for smaller companies or startups to access such compute. The move comes as Microsoft and… 18 llama.cpp releases dev-tools 7d ago b11466 ggml-webgpu: fix flash_attn supports_op check for overlapping KV ( #28205 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53548008 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS… 17 r/LocalLLaMA community 7d ago Micron Says NVHBM to Improve Profitability Even With Outsourced Base Die "NVHBM moves the memory controller, which was previously located on the main compute die, into the base die. This reduces power consumption by 15% and increases memory bandwidth by as much as 30%. It also integrates a customized physical layer (PHY) for input/output (I/O),… 22 r/LocalLLaMA community 7d ago Image-text retrieval with EmbeddingGemma 2's vision tower, running in the browser on WebGPU EmbeddingGemma 2 came out this week. It maps images and text into one 768-dim space, so you can search photos by describing them. I ported its text and vision towers to ruNNtime, a WebGPU inference library in TypeScript, and made a small photo gallery where search runs entirely… 28 r/LocalLLaMA community 7d ago I re-trained the DFlash 2 drafter for Ternary Bonsai 2 27B: 2.2x on an L4 (3.2x on code edits with ngram lookup), 1.5x on a Mac, 1.2x in Chrome PrismML's Ternary Bonsai 2 27B fits a 24 GB card or Mac, but decodes at ~30 tok/s on an L4 and ~21 on an M4 Pro. z-lab's DFlash 2 drafter was trained on bf16 Qwen3.8-27B, so it guesses worse on the ternary model. I fine-tuned it on 1.5M tokens of Bonsai 2's own greedy output.… 22 llama.cpp releases dev-tools 7d ago b11464: [SYCL] fix the issue in mixed different model GPUs in FA (#29071) fix mixed different model GPUs issue Update ggml/src/ggml-sycl/ggml-sycl.cpp Co-authored-by: Georgi Gerganov ggerganov@gmail.com Co-authored-by: Georgi Gerganov ggerganov@gmail.com 12 r/LocalLLaMA community 7d ago Ramjet - mini altermative to nvidia dynamo Hello, if you are running multi gpu setup, check out ramjet https://github.com/helixml/ramjet The goal is to have a local version of dynamo that can do equal or better job (but ideally without k8s). Check out https://helix.ml/blog/ramjet-vs-nvidia-dynamo as well. If you have dgx… 4 llama.cpp releases dev-tools 7d ago b11461 vulkan: fix amd iGPU slow checkpoint read ( #30049 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53465294 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS… 32 llama.cpp releases dev-tools 7d ago b11459 ggml-webgpu: no dawn native features on wasi ( #27069 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53445261 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS… 9 arXiv — Machine Learning research 7d ago Event-Driven ML Pipeline Orchestration for Manufacturing: An AWS Industry Experience arXiv:2610.06890v1 Announce Type: new Abstract: We present an industry experience report on three years of operating an event-driven cloud infrastructure for continuous machine learning training in automotive manufacturing. Our system orchestrates GPU-accelerated training of… 5 arXiv — Machine Learning research 7d ago When Attention Does Not Explain the Peak: Temporal Reference vs. Forecast Output in Attention-Based Time-Series Forecasting arXiv:2610.07080v1 Announce Type: new Abstract: Attention maps are often interpreted as evidence of what a forecasting model uses when making predictions. In our load-forecasting model, a CLS representation of historical demand queries 24 future exogenous horizon tokens through… 15 arXiv — Machine Learning research 7d ago AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation arXiv:2610.07457v1 Announce Type: new Abstract: Fine-grained mixed-precision quantization promises efficient large language model inference, but local precision choices can conflict with regular GPU storage and computation units. This precision-boundary mismatch limits the… 4 arXiv — NLP / Computation & Language research 7d ago Stabilizing language models under continual learning via condition-anchored distillation arXiv:2610.06940v1 Announce Type: new Abstract: Continual adaptation of language models can change their output distribution on prompts learned earlier, while retaining every old prompt-answer pair may be undesirable or impossible. We study condition-anchored generative… 24 arXiv — NLP / Computation & Language research 7d ago JudgeMoE: Distributional Aggregation for LLM-as-a-Judge arXiv:2610.07109v1 Announce Type: new Abstract: When an LLM judge scores an output, its score distribution retains uncertainty and disagreement information that is lost after scalar compression. We introduce JudgeMoE, a lightweight aggregator that assigns example-specific… 19 arXiv — NLP / Computation & Language research 7d ago Hybrid Latent Attention for Looped Language Models arXiv:2610.07940v1 Announce Type: new Abstract: Looped language models apply the same stack of layers T times to each token, which deepens the model without adding parameters but multiplies its key-value (KV) cache by T. The larger cache limits how many sequences a GPU can… 24 arXiv — NLP / Computation & Language research 7d ago Are Language Models Script-Aware? arXiv:2610.08037v1 Announce Type: new Abstract: Language models frequently generate outputs in unintended languages or scripts, a phenomenon known as off-target generation. While existing research has focused on language selection, the dimension of script knowledge remains… 20 arXiv — NLP / Computation & Language research 7d ago Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment arXiv:2610.07023v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable capabilities but remain vulnerable to jailbreak attacks that elicit harmful or unsafe outputs. Existing safety alignment approaches, including Supervised Fine-Tuning (SFT) and… 18 r/LocalLLaMA community 7d ago Why is ik_llama.cpp said to be faster than Mainline? On my hybrid multi-GPU rig, Mainline easily beats it I constantly see recommendations saying that ik_llama.cpp (ikawrakow's fork) is the undisputed king of hybrid CPU/GPU offloading and MoE performance. However, every time I benchmark it against mainline ggml-org, mainline consistently beats it by a wide margin. Am I missing… 28 The Information — AI news-outlet 7d ago SpaceX Seeks to Raise $40 Billion From Apollo to Buy Nvidia Chips SpaceX, the AI and rocket company led by Elon Musk, is seeking to raise $40 billion to buy Nvidia chips in a financing round led by Apollo Global Management, the Financial Times reported on Tuesday. The deal is expected to close in 2027 and consist of around $10 billion in bank… 23 r/LocalLLaMA community 7d ago Ruach Studio: a whole song studio around YuE2 on your own GPU. Score first, LoRA training, stems,remaster, DAW export. We spent the last weeks building a studio around YuE2 , the open song model by m-a-p, and today it reaches its first release candidate. Write the style and the lyrics, and it composes, sings and renders the song on your own card. Nothing leaves your machine unless you point the… 23 llama.cpp releases dev-tools 8d ago b11457 cuda : add BF16 support for XIELU ( #29955 ) The XIELU CUDA kernel template is already generic over the element type; only the F32/F16 type assertion and the else-if dispatch were missing. Add the nv_bfloat16 branch to the launcher, and drop the temporary supports_op gate in… 10 Page 3 of 10 · 500 articles ← Newer Older →
