News / #gpu Tag Gpu 500 articles archived under #gpu · RSS Sign in to follow r/MachineLearning community 3d ago [P] Pecision models that score every allowed label from the logits: Jebadiah v2.1 (27B, 9B), open weights and self-run benchmark results [P] I've been building open models that treat a decision as a closed-set scoring problem rather than text generation. The input is structured context plus a typed question with a fixed set of options. The output is a probability for each option, taken from the candidate-label… 23 r/LocalLLaMA community 3d ago Local Voice Assistant based on Qwen3.5 4B with skills/toolcalling. No dedicated GPU. Local STT+TTS STT: Voxtral TTS: Pocket Longer video: https://www.youtube.com/watch?v=1WmvMO-f-24   submitted by   /u/Leading_Yogurt7025 [link]   [comments] 13 r/LocalLLaMA community 3d ago Imagine M5 Ultra + this: NVIDIA GeForce RTX 5060 runs macOS 15 with Metal 3 acceleration through unofficial driver   submitted by   /u/chemist_slime [link]   [comments] 13 r/LocalLLaMA community 3d ago Be careful fam - New RTX 4090 fakes hit the market, with GPUs and memory almost indistinguishable from real chips Be careful out there of deals now. If it’s too good to be true…   submitted by   /u/chemist_slime [link]   [comments] 23 r/LocalLLaMA community 3d ago AMD Reportedly Raises GDDR6 Prices for Board Partners Is an AMD price incoming as well? Nvidia just discontinued the 5090 in favor of RTX Pros   submitted by   /u/chemist_slime [link]   [comments] 11 r/LocalLLaMA community 3d ago Building a 4x R9700 setup for a 10 person startup Just wanted to share a build I am doing for a client. $18k. Specs: Threadripper 9970x 128gb DDR5 ECC 5600 4x AMD Radeon R9700 AI Pro 32gb(128gb total VRAM) 1600W PSU(GPUs are undervolted and under 210w each) Engine : using a fork of Radiance to serve Qwen 3.8 Next flash/27b and… 22 r/LocalLLaMA community 4d ago No more RTX 5090 Nvidia reportedly halts GeForce RTX 5090 production in favor of AI data center and professional GPUs — impending supply drought expected to drive up prices, RTX 5080 24GB rumored as new gaming flagship… 4 The Information — AI news-outlet 4d ago GPUs in the Wine Cellar: Why Techies Are Hoarding AI Compute in Their Homes The basement of Gary Flake’s contemporary home in a quiet neighborhood in Bellevue, Wash., is furnished with everything you’d expect in a multipurpose man cave. There’s a cozy-looking couch situated between a large screen and a digital projector, a Peloton treadmill and a rowing… 6 The Information — AI news-outlet 4d ago GPUs in the Wine Cellar: Why Techies Are Hoarding AI Compute in Their Homes The basement of Gary Flake’s contemporary home in a quiet neighborhood in Bellevue, Wash., is furnished with everything you’d expect in a multipurpose man cave. There’s a cozy-looking couch situated between a large screen and a digital projector, a Peloton treadmill and a rowing… 23 r/LocalLLaMA community 4d ago Qwen3.8-27B on a single 3090: 140 tok/s on code with a custom megakernel I've been using Claude Opus 5.5 to speed up Qwen3.8-27B on my PC (rtx 3090), it wrote a CUDA megakernel that is 1.4-1.9x faster than llama.cpp depending on the task/context length. The results and code are below: Results (same hardware, my megakernel vs llama.cpp with MTP): -… 16 r/LocalLLaMA community 4d ago NVIDIA reportedly discontinuing RTX 5090, GB202 GPUs to be reserved for RTX PRO series No.....   submitted by   /u/chemist_slime [link]   [comments] 30 r/LocalLLaMA community 4d ago Strata with Qwen3.8 Flash Next UD-Q4_K_XL Been seeing quite a few posts about Strata lately, so I figured I'd give it a shot on my RTX PRO 6000 and I am very impressed. Most of the benchmarks I've seen are using IQ2 or IQ3 quants, so I wanted to see how Unsloth's UD-Q4_K_XL performs instead. Setup: GPU: 1x RTX PRO 6000… 35 r/LocalLLaMA community 4d ago microsoft doubles down on local ai with nvidia, but with a cost https://preview.redd.it/uyjw2lcrvkuh1.png?width=1536&format=png&auto=webp&s=ce991be22faec3de5cbaf39119c14782b6ed0af3 link to article TLDR; new microsoft surface and nvidia rtx spark laptops are starting at 2.6k and hitting nearly 7k for the 128gb unified memory models. local llm… 5 llama.cpp releases dev-tools 4d ago b11546 cuda: always use MMVQ for MUL_MAT_ID on sm_60 ( #27828 ) 18 llama.cpp releases dev-tools 4d ago b11545 CUDA/HIP: fix race in flash_attn_ext_f16_process_tile when nbatch_com… 24 llama.cpp releases dev-tools 4d ago b11543: vulkan: use 4 rows for NVIDIA MUL_MAT_ID MMVQ (#29274) Keep the existing RDNA3/4 policy unchanged and update only rm_id to use 4 rows for NVIDIA except pre-Turing. 17 r/MachineLearning community 4d ago Real-time neural weather restyling for Minecraft on a GTX 1650 (1.4M-param U-Net, distilled from FLUX.2 klein, 30-40 FPS realtime on budget GPU) [P] Teacher: FLUX.2 klein 4B paints ~3k frames (snow / wet / night at 3 strengths). Student: 1.4M-param U-Net with FiLM sliders, 512×288, ~26 ms/frame via ONNX Runtime inside a Fabric mod; the HUD stays untouched. Pixel loss (L1+MSE) gave a washed-out “average” because the teacher… 20 r/LocalLLaMA community 4d ago SlopSoup TV. A live, never-ending 24/7 pixel-art TV network inspired by old-school late night Adult Swim. No human makes any of it. SlopSoup TV is a live, never-ending pixel-art TV network. Every script, character, voice, camera cut and schedule decision is made by models. It's made to be very absurd, dark and strange. Hardware: one Hugging Face Space, 16 vCPU, no GPU. Everything below runs on CPU next to a… 31 r/MachineLearning community 5d ago Talus: a 23M-parameter diffusion model for game terrain, evaluated against a real-vs-real noise floor, running in the browser on WebGPU [P] Talus generates 64x64 heightmaps (4 km, up to 1,200 m) conditioned on a terrain type and any subset of five measured properties: mean elevation, relief, mean slope, water fraction and spectral slope. I trained it from scratch on one RTX 5060 (8 GB), about 4.5 hours in total… 16 r/LocalLLaMA community 5d ago Qwen 3.8 Flash Next-GSQ-RCO-IQ2_XS at ~21 tok/s on just an RTX 3060 12GB + 16GB DDR4 RAM(No gate pruning, 100% bit-exact) (Don't judge by the screenshot, the cache is cold. It hits 24+ tok/s with a warm cache!) About two months ago, I made a post here asking whether predicting which MoE experts would be used on the next token could actually help speed up CPU/GPU offloading. Original post: Tried… 11 llama.cpp releases dev-tools 5d ago b11538 CUDA: fix round issue, under MSVC the CPU and GPU agree ( #30229 ) 31 llama.cpp releases dev-tools 5d ago b11533 opencl: fix kernel compilation for a6x GPUs ( #30176 ) opencl: skip kernel_cpy_f32_f32_pack on A6X to avoid shader compiler crash The A6x compiler backend found in iot device with a623 (E031.50.31.01) cannot handle kernels with a large number of arguments. Skip this kernel for… 4 llama.cpp releases dev-tools 5d ago b11534: CUDA: Remove redundant CUDA copies after SSM_SCAN (#29807) CUDA: fuse copy of updated state snapshots into recurrent cache with ssm_scan CUDA: remove redundant cuda copies with K==1 (non spec-dec) scenario as well 33 llama.cpp releases dev-tools 5d ago b11530 llama: keep the backend sampling graph static across ubatches ( #30223 ) The reserve builds n_outputs_max_per_seq sampling chains per sampler, while a decode built one per output row, so the graph changed its topology after the reserve and GGML_SCHED_NO_REALLOC builds aborted on… 24 r/LocalLLaMA community 5d ago GitHub - google-ai-edge/ml-drift: GPU-Accelerated AI/ML Inference Blog Post : ML Drift: Next-Gen GPU AI/ML Inference at the Edge - Google Developers Blog The Google AI Edge Team is excited to announce the open-source release of ML Drift , our high-performance, cross-platform, on-device GPU compute engine specifically built for on-device AI/ML… 29 Hugging Face official-blog 5d ago Impactful scheduling for GPU clusters Back to Articles a]:hidden"> Impactful scheduling for GPU clusters Enterprise Article Published October 9, 2026 Upvote - Kyle Wiggers Ai2Comms allenai :last-child]:mb-0"> Building a cluster scheduler to prioritize high-impact research while maintaining full occupancy On the AI… 6 llama.cpp releases dev-tools 5d ago b11527 webgpu: use 2D workgroup dispatch for all the ops which use 1D dispat… 33 llama.cpp releases dev-tools 5d ago b11523 ci: fix Models Backend webgpu by supporting GGML_OP_DUP ( #30216 ) The mixed batch path of PR-29622 writes the token rows with set_rows into a dup of the embeddings. WebGPU did not support DUP, so the dup ran on the CPU while the set_rows writing into it was scheduled on WebGPU,… 32 llama.cpp releases dev-tools 5d ago b11517: CUDA: pass src1 precision to host MMQ config helpers (#30168) CUDA: pass src1 precision to host MMQ config helpers Signed-off-by: ynankani ynankani@nvidia.com make src1 prec explicit Signed-off-by: ynankani ynankani@nvidia.com Signed-off-by: ynankani ynankani@nvidia.com 31 llama.cpp releases dev-tools 5d ago b11516: hexagon: fix IM2COL patch-embed DMA ring overflow (#30189) hexagon: fix IM2COL patch-embed DMA ring overflow The exact-tiling (stride == kernel, no pad/dilation) IM2COL DMA kernel issues IC KH DDR->VTCM descriptors per output row without checking the return value of dma_queue_push(), and then pops IC KH times. The per-thread DMA ring… 37 The Information — AI news-outlet 5d ago Nvidia-Backed Firmus Scraps $5 Billion Australia IPO as Markets Wobble Firmus Grid, an AI cloud company backed by Nvidia, has pulled what would have been Australia’s biggest initial public offering in almost 30 years. The Sydney-based company had planned to raise nearly $5 billion at a valuation above $30 billion. It cited market volatility and… 31 arXiv — Machine Learning research 5d ago When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry arXiv:2610.10616v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) language models produce routing information during inference that may be logged or exposed for monitoring, debugging, load analysis, and safety auditing. Unlike ordinary model outputs, this telemetry… 16 arXiv — Machine Learning research 5d ago CARE: A Lightweight Plug-in Gated Correction and Uncertainty-aware Module for Long-term Time Series Forecasting arXiv:2610.11165v1 Announce Type: new Abstract: Multivariate long-horizon forecasting is critical to electricity load scheduling and traffic flow management, and to financial risk control. Existing deterministic backbones output a single trajectory, masking heterogeneous… 32 arXiv — NLP / Computation & Language research 5d ago MetaEncoder: Exploring the Limit of Bi-Encoders for Multimodal System One Decision Making with Natural Language Interface arXiv:2610.11316v1 Announce Type: new Abstract: System One models output constrained decisions and probability distributions rather than free-form text generation. While prevailing paradigms rely on structured schema objects to encode state, intent, and candidate choices, we… 5 arXiv — NLP / Computation & Language research 5d ago Fact over Fiction: Detection of Pathological Hallucinations in Sinhala-to-English Neural Machine Translation arXiv:2610.11389v1 Announce Type: new Abstract: Neural Machine Translation (NMT) models, while capable of producing highly fluent outputs, remain vulnerable to hallucinations, which are translations that are natural yet semantically unrelated to the source. This vulnerability is… 31 arXiv — NLP / Computation & Language research 5d ago Constitutional Gating and Deterministic Recovery for Multi-Agent LLM Negotiation: Ablations Against a Stateful Adversarial Gatekeeper arXiv:2610.11542v1 Announce Type: new Abstract: Multi-agent LLM systems negotiating with a stateful counterpart waste model calls in three ways: polite loops that never meet the counterpart's hidden acceptance condition, malformed outputs that trigger retries, and compliance… 35 arXiv — NLP / Computation & Language research 5d ago Same Outcome, Different Evidence: Intent Recovery in LLM Safety Evaluation arXiv:2610.11766v1 Announce Type: new Abstract: Safety evaluations of large language models commonly summarize harmful-output behavior with attack success rate (ASR). Yet the same non-harmful outcome can arise for very different reasons. A model may recover a harmful task and… 26 arXiv — NLP / Computation & Language research 5d ago Easy to anticipate, hard to compute: boundary dependence finds the computed outputs that entropy patching misses arXiv:2610.11790v1 Announce Type: new Abstract: Byte-level language models such as the Byte Latent Transformer (BLT) group bytes into patches and run their large global model once per patch. BLT starts a patch where a small model's next-byte entropy is high, so global compute… 9 arXiv — NLP / Computation & Language research 5d ago WorldBench: Evaluating LLMs on Three.js Voxel World Generation arXiv:2610.10622v1 Announce Type: cross Abstract: Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges take one view of the output: a vision-language model scores a few rendered… 35 r/LocalLLaMA community 5d ago Qwen3.8-27B: 159 tok/s on R9700, 64 tok/s on Strix Halo LemonSeed Studio is an iPad editor/IDE with on-device inference on an AMD GPU in a Thunderbolt enclosure. It embeds the unmodified upstream Linux amdgpu + amdkfd driver (mac_linuxgpu) as a PCIDriverKit extension, and runs LemonSeed Engine (LSE) on the GPU. In the photos: iPad… 36 The Information — AI news-outlet 5d ago Nvidia to Invest in Chip Rival d-Matrix as Its Challengers Choose Partnership Nvidia plans to invest in d-Matrix, a seven-year-old developer of server chips for AI that aims to compete with Nvidia, according to three people with knowledge of the deal. The planned investment is part of Nvidia’s effort to make its technology compatible with rival chip… 35 NVIDIA Developer Blog official-blog 6d ago Building Reliable Data Analytics Agents: Lessons from the KDD Cup The NVIDIA KGMON team placed second in the KDD Cup 2026 Data Agents competition with a system built around a simple idea of making an agent's harness smaller,... 30 llama.cpp releases dev-tools 6d ago b11512 model : fix DFlash output head sharing ( #30111 ) llama : fix DFlash output head sharing Assisted-by: Codex dflash : read tied output weights from GGUF metadata Assisted-by: Codex llama : share tied word embedding metadata Assisted-by: Codex llama : remove DFlash embedding head… 23 r/LocalLLaMA community 6d ago Running decision model locally on an RTX 4090 to find out which one is the fastest recently saw a bunch of open decision models pop out of nowhere in the last two weeks (laya, liquid's d1, cloudflare's clef-flash, interfaze's lev), so I wanted to see how far apart they actually are on the same GPU(yes, model size is a huge factor, but still isn't the only… 35 Hacker News — AI on Front Page community 6d ago OpenAI annualised revenues $20B less than previously signalled Article URL: https://www.cnbc.com/2026/10/08/open-ai-revenue-nvidia-oracle-coreweave.html Comments URL: https://news.ycombinator.com/item?id=50008187 Points: 223 # Comments: 130 18 llama.cpp releases dev-tools 6d ago b11513: CUDA: improve top-k algorithm selection (#28713) CUDA: radix top-k for large row counts Replaces CUB's per-row DeviceTopKKernel with a grid-over-rows radix select, gated on GGML_CUDA_TOPK_RADIX_MIN_ROWS. On qwen4exp at 34,816 tokens this cuts top-k from 1,671,253 launches / 5,761.8 ms to 2,329 / 941.8 ms. CUDA: select the… 31 llama.cpp releases dev-tools 6d ago b11509 CUDA: fix CCCL version guard breaking on major version rollover ( #29453 ) CUDA: fix CCCL version guard breaking on major version rollover The guard compared the major and minor components independently: CCCL_MAJOR_VERSION >= 3 && CCCL_MINOR_VERSION >= 1 Minor resets to 0… 24 llama.cpp releases dev-tools 6d ago b11507 llama: support MoE cache over multiple GPUs ( #30112 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53988921 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS… 29 llama.cpp releases dev-tools 6d ago b11511 CUDA: fix MMQ out-of-bounds reads ( #29953 ) 19 llama.cpp releases dev-tools 6d ago b11510 CUDA : looped PAD kernel for more than 65535 rows or slices ( #30147 ) 34 Page 2 of 10 · 500 articles ← Newer Older →
