News / #gpu Tag Gpu 500 articles archived under #gpu · RSS Sign in to follow r/LocalLLaMA community 15d ago Building an open-source control plane for self-hosted vLLM, what would you want in it? Every time I self-host a model I rebuild the same stuff: start the container, set up a route, check why it died overnight, remember to shut the GPU off before it burns money. So I'm building a panel that handles it. Start/stop models, OpenAI-compatible endpoint, health checks… 25 r/LocalLLaMA community 15d ago GPU prices haven't stopped climbing for 3 weeks straight across the EU, here's the data hey again! I run a EU PC hardware price tracker PriceSquirrel , 25+ stores across 9 countries, and wanted to check: are GPU prices actually rising, or does it just feel that way? To make this defensible, I didn't just average "whatever's in stock" each day, that inflates the… 13 r/MachineLearning community 15d ago If you had a bunch of GPUs lying around, what would you actually build with them? (Running LLMs is off the table) [D] Be honest if someone dropped a stack of high-end GPUs on your desk tomorrow, what would you actually do with them? And before the usual answers roll in: running local LLMs is banned for this thread. It’s been done to death and feels pretty pointless at this point. So… what else?… 28 r/LocalLLaMA community 15d ago GitHub - meta-models/meta-oss-cookbook: All recipes for oss models from Meta Inc. Muse Glimmer Cookbook Clone it, run it on the GPU you already own, and ship a working agent — one that plans, calls tools, and self-corrects — in a single sitting, fully offline. Muse Glimmer is an open-weight model built for local agentic work on a single GPU. This cookbook is… 24 r/LocalLLaMA community 15d ago Qwen3.8-27B Q6_K at 128K on a single 32GB GPU Fresh download today. Really quick: my impression of Qwen3.8-27B: I ran the Q6_K GGUF locally on a 32GB R9700 through llama.cpp + OpenCode, with 128K context. Q8 would not fit with enough left over for context. I gave it my real, years-old swimming pool-controller repository and… 16 llama.cpp releases dev-tools 15d ago b10431 ggml : recurrent state rollback for ggml_ssm_scan ( #26623 ) Initial changes for Recurrent state rollback for nemotron for cpu and cuda Removing CPU RS rollback. Will enable it in subsequent PRs addition of test case Removing assert and calling runtime API to check if op is… 21 r/LocalLLaMA community 15d ago *Cries in 18GB unified memory* Equivalent to 14GB of VRAM and 4GB of system RAM given the limits I assigned to the GPU. EDIT: we be getting lots of big (≥20B), small (≤4B) models these days but very little in between   submitted by   /u/banana_slurp_jug [link]   [comments] 25 TechCrunch — AI news-outlet 15d ago Kog is going deeper to squeeze more inference out of GPUs The idea that GPUs are poorly suited for agentic workflows may be a misconception, according to French startup Kog. 9 r/MachineLearning community 15d ago A linter for PyTorch 'torch-preflight' [P] Been working on this for the last few months. I've been working on PyTorch for the past few years and I always felt, many a times my work went into dump, because of some mistakes I made in the code. torch-preflight reads your PyTorch code and catches the bugs costing you GPU… 15 llama.cpp releases dev-tools 16d ago b10430 llama : allow virtual igpu devices ( #26953 ) llama : allow virtual igpu devices cont : better comment Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64… 26 r/MachineLearning community 16d ago Building text to ASCII diffusion model , need advice and guidance [P] i wanna build a text diffusion model which interpret text and convert it into ascii images so like Text : build a cat Output : /\\\_/\\ ( o.o ) \> \^ < So , i have a decent background of ml algo ( completed cs229 , cs230 , Ml architecture and basic CNN and diffusion model ) ik… 37 llama.cpp releases dev-tools 16d ago b10424 dflash : clarify output logging of target_layer_ids ( #27013 ) This commit tries to make the logging of target_layer_ids a bit clearer and easier to read. Currently the output generated looks like this: 0.00.468.624 D load_arch_hparams: DFlash extract_layers = [0.00.468.626 D 2,… 25 arXiv — Machine Learning research 16d ago CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution arXiv:2608.12629v1 Announce Type: new Abstract: GPU kernel agents and GPU programming languages have advanced separately, leaving expert kernels difficult to reproduce. Agents usually treat the compiler as a fixed black box and receive only errors, correctness outcomes, and… 11 arXiv — Machine Learning research 16d ago A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family arXiv:2608.12700v1 Announce Type: new Abstract: Systems that generate GPU kernels with language models report high correctness rates. Those rates come from a single loose test: run the kernel on a few random inputs at one fixed shape and accept it if the output is close to a… 20 arXiv — NLP / Computation & Language research 16d ago The "Knowledge-Behavior Gap" in Cultural Taboo Safety of Large Language Models arXiv:2608.12341v1 Announce Type: new Abstract: Cultural taboo safety is essential for deploying large language models (LLMs), as culturally insensitive outputs may cause offense or even social harm. However, existing cultural benchmarks primarily assess cultural knowledge or… 11 arXiv — NLP / Computation & Language research 16d ago Novels generated by language models show compressed formal variation arXiv:2608.12630v1 Announce Type: new Abstract: While large language models can generate entire novels, there is little information about the level of formal variation in their output over many generations. Rather than asking whether individual passages can be identified as… 33 arXiv — NLP / Computation & Language research 16d ago Prompts in the Wild: A Large Analyzed Collection of Transactional Prompts in Code arXiv:2608.12905v1 Announce Type: new Abstract: The behavior of contemporary generative Large Language Models (LLMs) is directly shaped by prompts, unstructured texts that describe the desired output and model behavior. In this paper we argue that prompts are linguistic objects… 35 arXiv — NLP / Computation & Language research 16d ago Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction arXiv:2608.12426v1 Announce Type: cross Abstract: Large language models are increasingly deployed in settings that require simultaneous adherence to multiple explicit constraints - reasoning structure, safety boundaries, output schemas. Individual constraints are handled… 33 arXiv — NLP / Computation & Language research 16d ago TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes arXiv:2608.13057v1 Announce Type: cross Abstract: In expert-parallel (EP) MoE serving, every layer synchronizes at the slowest GPU. Dispatchers balance token counts (EPLB, LPLB, UltraEP) or activated-expert counts (METRO), assuming expert time is linear in one. Measurements on… 32 ThursdAI news-outlet 16d ago ThursdAI - Grok 4.6, Grok Bot deep dive, DeepSeek v4 Pro, Meta Muse Glimmer & more AI news | ThursdAi Aug 13 From Coreweave: Listen to this weeks ThursdAI with guests from NVIDIA, Artificial Analysis and Cursor/SpaceXAI 23 Anthropic SDK (Python) releases dev-tools 16d ago v0.122.0 0.122.0 (2026-08-13) Full Changelog: v0.121.0...v0.122.0 Features api: add output_behavior to dream creation (create a new memory store or update the input store in place) ( 852c4bb ) Bug Fixes bedrock,aws: run SigV4 signing off the event loop in async clients ( #334 ) ( 2bae6c8… 23 TechCrunch — AI news-outlet 16d ago Nvidia’s new $500B plan is risky but brilliant, especially for aging GPUs Nvidia has a plan to make sure its GPUs won't lose value. It wants to convince a new crop of financiers to keep lending for AI buildouts. 30 OpenAI official-blog 17d ago Previewing Ultrafast mode: GPT-5.6 Sol at up to 14X the speed Preview Ultrafast, a new OpenAI API service tier that runs GPT-5.6 Sol up to 14× faster. Powered by Cerebras, it delivers up to 750 output tokens per second. 24 Hugging Face Daily Papers research 17d ago Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control Abstract Two studies define measurable GPU control gates for LLM-agent services by analyzing concurrent cohort scheduling and on-device routing versus host redispatch. Generated by thinkingmachines/Inkling-Small LLM-agent services repeatedly execute small deterministic… 25 r/LocalLLaMA community 17d ago Nvidia doubles RTX PRO 6000 Blackwell's MSRP to a staggering $16,000 — 96GB card started pre-orders below $8,000 last year Given Gavin Bakers recent interview where he says: > Multiple private companies are planning on spending at least 2x more per GPU for compute as contracts roll-off and some have spoken about this publicly. https://x.com/i/status/2082166566280642676 How long before DGX Sparks… 20 arXiv — Machine Learning research 17d ago Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads arXiv:2608.11661v1 Announce Type: new Abstract: A multiplicative dual-encoder network computes a real-valued output for a pair of inputs as the inner product of their separate encodings. This architecture has been developed independently in operator learning, bipartite matching,… 26 arXiv — Machine Learning research 17d ago Kernel Methods for Learning Operators with Multiple Inputs and Outputs arXiv:2608.11831v1 Announce Type: new Abstract: Learning mappings between infinite-dimensional objects is a central challenge in scientific machine learning. We introduce a general kernel-based encoder-decoder framework for operator learning that separates observation,… 12 arXiv — Machine Learning research 17d ago A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression arXiv:2608.11917v1 Announce Type: new Abstract: Multi-output Gaussian process regression scales cubically in the number of observations times outputs, and dense kernel-matrix methods need bespoke handling whenever different outputs are observed at different inputs. We express… 36 arXiv — Machine Learning research 17d ago SoftWater: Class-Aware Rate Allocation for Softmax Quantization arXiv:2608.12026v1 Announce Type: new Abstract: Post-training quantization pipelines routinely leave the softmax output layer in high precision. Yet in small LLMs with modern vocabularies, the head holds 15--30\% of all parameters, so a nominal ``2-bit'' model with an fp16 head… 27 arXiv — NLP / Computation & Language research 17d ago Backtrader-Bench: Benchmarking LLM Agents on Algorithmic Trading with Self-Generated MCQs arXiv:2608.11232v1 Announce Type: new Abstract: Evaluating LLM coding agents in algorithmic trading is difficult because static benchmarks risk data contamination and numerical backtest outputs require ground truth from actual code execution. We present Backtrader-Bench, a… 18 arXiv — NLP / Computation & Language research 17d ago Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning arXiv:2608.11408v1 Announce Type: new Abstract: Prior white-box studies show that large language models can retain latent traces of target knowledge after unlearning, even when the knowledge is no longer expressed in their outputs. However, existing audits remain limited to… 25 arXiv — NLP / Computation & Language research 17d ago Is Convergence Inevitable? Tracing Output Homogeneity Back to Base Models arXiv:2608.11426v1 Announce Type: new Abstract: The lack of diversity in LM content is widely attributed to the alignment process, but how and where exactly in the pipeline this collapse begins is unknown. We argue that output homogeneity is likely learned during the pretraining… 12 arXiv — NLP / Computation & Language research 17d ago Beyond Single-Turn Confidence: Trajectory-Adapted Uncertainty Quantification for LLM Agents arXiv:2608.11552v1 Announce Type: new Abstract: Uncertainty quantification (UQ) methods for language models are typically evaluated on single-turn outputs, where uncertainty is attached to one generated answer. For LLM agents, however, the unit of observation is an interactive… 4 arXiv — NLP / Computation & Language research 17d ago Locating and Controlling Implicit Personalization in Large Language Models arXiv:2608.11735v1 Announce Type: new Abstract: Large language models (LLMs) often shift their outputs in response to implicit demographic cues even when users never state a demographic identity. Previous work has documented this behavior, but the connection between these… 8 arXiv — NLP / Computation & Language research 17d ago LazyTrain: Limited-resource Allocation toward Zero-waste Yield Optimization in Large Language Model Training arXiv:2608.11919v1 Announce Type: new Abstract: Training large language models on limited hardware is increasingly a scheduling problem across GPU compute, host memory, PCIe transfer, and storage bandwidth. Existing offloading systems reduce GPU residency, and MegaTrain shows… 19 r/LocalLLaMA community 17d ago Qwen3.6 35B (2 min) vs Muse Glimmer 30B (4 min) on custom Llama.cpp build (RTX 5080) Muse Glimmer 30B feels significantly more precise and reliable, it almost never drops the ball or breaks rules. However, its designs lack creative depth and richness. Qwen3.6 35B, on the other hand, is prone to more occasional blunders/hallucinations, but its creative output is… 35 Ollama releases dev-tools 17d ago v0.32.10-rc0: nn: speed up prefill on double-scale nvfp4 models ModelOpt checkpoints apply a float32 global scale to every projection output on top of the per-group quantization scales. Running the multiply and the cast back to the activation dtype as separate eager ops costs an extra kernel launch and a materialized intermediate per… 14 NVIDIA Developer Blog official-blog 17d ago Serve Qwen3.8-2.4T-A95B, a 2.4T-Parameter Model, with Configurable Reasoning on NVIDIA GB300 NVL72 Alibaba released the open weights for Qwen3.8-2.4T-A95B (Qwen3.8-Max), its largest open-weight model, bringing near-frontier capabilities to the open... 35 NVIDIA Developer Blog official-blog 17d ago How to Choose Full-Stack Observability for NVIDIA AI Factories AI infrastructure spans multiple layers, from compute and networking to storage, orchestration, and applications. When performance degrades, identifying the... 34 r/LocalLLaMA community 17d ago Best models 14b and smaller as of today? For the GPU impoverished   submitted by   /u/Thatisverytrue54321 [link]   [comments] 10 r/LocalLLaMA community 18d ago NVIDIA's Fastest Blackwell GPU, the 96 GB RTX PRO 6000, Now Costs $16,000, Almost Double Its Original Price   submitted by   /u/ab2377 [link]   [comments] 6 r/LocalLLaMA community 18d ago According to AMD, Arm, and Microsoft, agentic AI could push CPU-to-GPU ratios from 1:4 to even1:1 In OCP APAC 2026, Tai AMD SVP of compute and enterprise AI said agents don't cut GPU demand but they just pile on a whole extra layer of orchestration, retrieval, and tool-calling work that runs on CPUs instead And the usual 1:4 CPU-to-GPU ratio could move toward 1:2 or even 1:1… 23 Hugging Face Daily Papers research 18d ago Articulated Object Reconstruction from Rest-State Observation Abstract A rest-state framework reconstructs articulated objects from a single closed configuration by fusing vision-language outputs into consistent part meshes and validating synthesized motion hypotheses via geometric consistency. Generated by thinkingmachines/Inkling-Small… 29 r/LocalLLaMA community 18d ago RTX 6000 PRO price raised to $16,000 USD on the Nvidia website   submitted by   /u/Norwood_Reaper_ [link]   [comments] 29 arXiv — Machine Learning research 18d ago Fisher8: Stabilizing Neural Heteroscedastic Regression via Output-Layer Fisher Geometry arXiv:2608.10374v1 Announce Type: new Abstract: Training neural networks to jointly predict mean and uncertainty estimates from noisy observations can be unstable, prompting a series of independent stabilization efforts. We argue that these interventions highlight a common… 16 arXiv — Machine Learning research 18d ago ProbGuard: Calibrated Safety Risk Estimation from LLM Output Distributions arXiv:2608.10621v1 Announce Type: new Abstract: Recent research on Large Language Model (LLM) safety has widely adopted guardrails to identify unsafe LLM outputs. Existing guardrails typically formulate safety assessment as a deterministic classification task, mapping a discrete… 37 arXiv — NLP / Computation & Language research 18d ago Mapping and Measuring the Behavioral Evolution of Large Language Models arXiv:2608.11027v1 Announce Type: cross Abstract: Benchmark leaderboards summarize how well a language model performs, but not how its behavior relates to that of other models or changes across generations. We characterize the output behavior of 32 models from six families using… 27 arXiv — NLP / Computation & Language research 18d ago The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding arXiv:2608.10137v1 Announce Type: new Abstract: Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step. However, rigid masking distorts the model's underlying probability… 16 arXiv — NLP / Computation & Language research 18d ago Mitigating Context Interference for Reliable and Efficient Search Agents arXiv:2608.10743v1 Announce Type: new Abstract: Recent research empowers Large Language Models (LLMs) as multi-turn search agents to iteratively retrieve and generate outputs until complex tasks are solved. However, the contexts of multi-turn search agents are lengthy and… 38 arXiv — NLP / Computation & Language research 18d ago What Iterated Self-Feeding Probes of Language Models Measure, and a test that separates the construction from the model arXiv:2608.10986v1 Announce Type: new Abstract: A growing class of methods probes a language model by feeding it its own output: self-consistency, iterated refinement, agentic loops. We ask what such a probe measures, in a construction chosen to make the question sharp: a ring… 5 Page 6 of 10 · 500 articles ← Newer Older →
