News / #gpu Tag Gpu 500 articles archived under #gpu · RSS Sign in to follow arXiv — NLP / Computation & Language research 11d ago PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX arXiv:2608.17379v1 Announce Type: new Abstract: We introduce PTXBench, a benchmark for evaluating and adapting large language models (LLMs) to use architecture-specific PTX for GPU kernel optimization. PTXBench measures functional correctness, whether selected target… 17 arXiv — NLP / Computation & Language research 11d ago Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See arXiv:2608.17744v1 Announce Type: new Abstract: Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark… 17 arXiv — NLP / Computation & Language research 11d ago Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees arXiv:2608.17994v1 Announce Type: new Abstract: Using LLMs as judges has become standard practice for evaluating model outputs at scale. This is particularly common for subjective, open-ended tasks such as assessing helpfulness or alignment, where no single reference answer… 25 Hugging Face Daily Papers research 11d ago Personalized Auto-Research: Towards a True AI Co-Scientist Abstract The paper introduces personalized auto-research, a framework that conditions AI-driven hypothesis generation, experimentation, and writing on individual researcher representations to avoid generic outputs. Generated by thinkingmachines/Inkling-Small AI co-scientists… 23 r/LocalLLaMA community 11d ago Ling-3.0-tiny is a very interesting model. Run on NVIDIA Orin Nano Super 8GB at 128K context with IQ4_NL quant. I have been searching for suitable model to run on my 8GB RAM toy, NVIDIA Orin Nano Super 8GB. This little toy was priced at $249 earlier this year (not any more), and pulls very little power when idle. It was an interesting device that suitable for an agent to host on. It is… 31 Hugging Face Daily Papers research 11d ago Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirements Abstract Agentic ESOpt uses evolution strategies for scalable full-parameter fine-tuning of long-horizon LLM agents via trajectory-level reward-weighted updates and parameter-context co-evolution. Generated by thinkingmachines/Inkling-Small Reinforcement Learning (RL) has been… 12 r/LocalLLaMA community 11d ago Alibaba's RISC-V CPU, XuanTie C950, Runs Qwen-3.8 27B at 30 tps Who needs GPUs?   submitted by   /u/DeltaSqueezer [link]   [comments] 25 NVIDIA Developer Blog official-blog 11d ago How AI Coding Agents Can Unlock Materials Simulation with NVIDIA ALCHEMI Toolkit Atomistic simulation requires three things: knowledge of the science, compute-efficient implementation of simulations, and accessible interfaces to the... 9 r/LocalLLaMA community 11d ago Qwen3.8 2.4T open weights made a Call of Duty clone Qwen released the 2.4T Max weights and I was curious how well it can re-create COD in one prompt I ran the model on a rented B200 cluster and used roughly 1.1M output tokens over a 5 hour time span Realistically barely anyone can run this model locally, but the fact that it's… 18 NVIDIA Developer Blog official-blog 11d ago Run Massive-Scale UMAP in Minutes Using Multiple GPUs—Without Losing Accuracy Uniform Manifold Approximation and Projection (UMAP) is a dimensionality reduction technique widely used for visualization and feature extraction. Applications... 16 r/LocalLLaMA community 11d ago Running DeepSeek V4 Flash Q4_K_XL at ~100 tok/s prompt processing on 4× RTX 3060 12GB I managed to run the 143–144 GiB DeepSeek-V4-Flash-0731 UD-Q4_K_XL GGUF on four RTX 3060 12GB cards while keeping a 360k–376k context window. Hardware: CPU: Intel Core i9-10920X, 12C/24T RAM: 128 GB DDR4-3200, quad-channel GPU: 4× NVIDIA RTX 3060 12GB Total VRAM: 48 GB Storage:… 25 Stratechery (Ben Thompson) community 12d ago Nvidia Backs OpenAI Data Center, Anthropic News, Google Buys Spirit Airlines Data Nvidia makes another deal, this time with a frontier lab; Anthropic's revenue continues to amaze; and maybe data finally is oil. 30 Hugging Face Daily Papers research 12d ago DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs Abstract Retired GPUs can form low-cost clusters for LLM inference, but their economic and environmental viability depends heavily on local electricity prices and carbon intensity. Generated by thinkingmachines/Inkling-Small As AI datacenters retire functional GPUs, vast… 9 llama.cpp releases dev-tools 12d ago b10481: CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark (#26843) CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark Signed-off-by: ynankani ynankani@nvidia.com skip moe experts and allow others based on k geometry (allow only small idle tail) Signed-off-by: ynankani ynankani@nvidia.com rename MMVQ DGX Spark params to GB10 and fix MSVC… 23 arXiv — Machine Learning research 12d ago DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs arXiv:2608.14614v1 Announce Type: new Abstract: As AI datacenters retire functional GPUs, vast quantities of still capable accelerators enter secondary markets. This paper investigates whether these retired GPUs can find a productive afterlife to form a DumpsterCluster that can… 16 arXiv — Machine Learning research 12d ago Paired Exact-Reset Evaluation of a Prediction-Derived Medium-to-Full World-Model Cascade arXiv:2608.14650v1 Announce Type: new Abstract: Existing adaptive-inference and world-action-model systems use cheap-stage outputs or predicted futures to allocate additional computation. We study a narrower question: under paired exact-reset physical outcomes, can a… 23 arXiv — Machine Learning research 12d ago Quantifying Depth Sufficiency in Residual Neural Networks: A First-Order Criterion arXiv:2608.14664v1 Announce Type: new Abstract: How can we determine whether a trained neural network is already deep enough? We study this under a fixed function-preserving residual-growth protocol specifying insertion locations, residual families, zero-output initializations,… 21 arXiv — NLP / Computation & Language research 12d ago HarmProfile: Characterizing Harmful Distributions in Frontier LLMs arXiv:2608.14577v1 Announce Type: new Abstract: Frontier large language models (LLMs) safety evaluation has largely treated harmful generation as an attack outcome rather than as an object of analysis. Consequently, little is known about the harmful outputs produced during model… 10 arXiv — NLP / Computation & Language research 12d ago DeMTS: Denoising Trajectories as Multivariate Time Series for Hallucination Detection in Diffusion Language Models arXiv:2608.14632v1 Announce Type: new Abstract: Diffusion large language models (D-LLMs) have emerged as a promising paradigm for text generation. However, similar to autoregressive LLMs, D-LLMs remain vulnerable to hallucinations, where fluent outputs may contain factually… 23 arXiv — NLP / Computation & Language research 12d ago Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Models arXiv:2608.14797v1 Announce Type: new Abstract: Large language models (LLMs) and large vision-language models (LVLMs) have demonstrated impressive generative capabilities, yet ensuring their outputs align with user intent is still challenging. While most existing approaches… 23 arXiv — NLP / Computation & Language research 12d ago LLMs Get Smarter from Targeted Synthetic Multilingual Data arXiv:2608.15964v1 Announce Type: new Abstract: Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt. In other words, a language model outputs different (and potentially incorrect) responses… 5 arXiv — NLP / Computation & Language research 12d ago Unadapted Multilingual ASR on a Garrusi Kurdish Evaluation Set: A Common-Reference Staged Normalization Analysis arXiv:2608.16379v1 Announce Type: new Abstract: Evaluating speech recognition for a Kurdish variety written in a Latin field orthography, using a model that outputs Arabic script, creates a measurement problem before a modelling one: direct scoring treats writing-system… 38 arXiv — NLP / Computation & Language research 12d ago Towards Computational Provenance: Carrying Causal-State Evidence in Generated Text arXiv:2608.16868v1 Announce Type: new Abstract: A language model's output does not by itself provide verifiable evidence about the internal computation that produced it. We study computational provenance: whether generated text can carry detectable evidence of which causally… 11 r/LocalLLaMA community 12d ago CDW has bumped the MSRP of the RTX Pro 6000 from $16,000 to $19,999 Did they slip up and leak future pricing? Live link: https://www.cdw.com/product/pny-nvidia-rtx-pro-6000-graphic-card-96-gb-gddr7/8326705 Archive link: https://web.archive.org/web/20260818013250/https://www.cdw.com/product/pny-nvidia-rtx-pro-6000-graphic-card-96-gb-gddr7/8326705… 19 Vercel — AI dev-tools 12d ago GLM 5.3 now available on AI Gateway GLM 5.3 from Z.ai is now available on AI Gateway. GLM 5.3 has improvements vs. GLM 5.2 at complex software engineering and at agent tasks that run across many steps, and it reaches those results while producing fewer output tokens than GLM 5.2 did at the same effort level. Z.ai… 30 OpenAI official-blog 12d ago How NVIDIA scales expertise with ChatGPT Work NVIDIA teams use ChatGPT Work to reduce manual tasks, connect fast-moving signals, and scale successful workflows globally. 13 Vercel — AI dev-tools 12d ago The Vercel Sandbox CLI is now more agent-friendly The Sandbox CLI version 4.0.1 ships four output and error improvements: New sandboxes point users towards how to connect into them. A sandbox not found error shows where to look for existing sandboxes. Common mistakes, like typing a command after sh , redirect to the command… 35 Latent.Space news-outlet 12d ago [AINews] Stripe buys OpenRouter for $7B No GPUs, no Agents, just really, really, really good infra and distribution. 5 NVIDIA Developer Blog official-blog 12d ago Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer Teams customize their models to hit their targets for latency, speed, memory, and compute. With the open NVIDIA Nemotron family of models, developers can find... 23 Hacker News — AI on Front Page community 12d ago GPU Offload in Rust: Portable, Safe, and Fast Article URL: https://arxiv.org/abs/2608.13759 Comments URL: https://news.ycombinator.com/item?id=49334991 Points: 225 # Comments: 46 19 llama.cpp releases dev-tools 12d ago b10472 cuda : skip UMA override for HIP builds ( #27083 ) AMD APUs report accurate memory via hipMemGetInfo. Using MemAvailable over-promises on small-carveout systems. fixes #18159 Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI… 28 TechCrunch — AI news-outlet 12d ago Groq raises $350M to fuel its pivot from AI chips to neocloud Groq raised $350 million at a $3.5 billion valuation as the former AI chipmaker pivots to a neocloud business and expands its Nvidia-powered data center footprint. 33 TechCrunch — AI news-outlet 12d ago Nvidia investing $1.5B in SoftBank data center developer behind OpenAI project Nvidia's investment in SoftBank's data center developer will guarantee its chips power an OpenAI data center. 13 Interconnects (Nathan Lambert) research 12d ago Teaching Everyone to Fish for Tokens Nvidia wants you building your own model, not buying from Anthropic/OpenAI. 35 r/LocalLLaMA community 12d ago EXL3 seems to be fading from the r/LocalLLaMa consciousness, and while I suspected it, I'm surprised at this point in time. EXL3 is an alternative to llama.cpp. And while there is extensive tooling for llama.cpp, EXL3's primary deployment ( TabbyAPI ), has a OpenAI compatible API so it shouldn't matter. Why won't this tool matter to you? If you have a GPU with under 24 GB of VRAM, the value kind of… 27 r/LocalLLaMA community 13d ago 100$ worth of gpu runs qwen 3.8 27b at 7.39 t/s Qwen 27b Q3_K_M 2x rx 580 8gb (~50$ each in my country, edge cases 60$ per gpu) gives us 16gb vram We used it on an old already existing ddr3 motherboard with 2 gpu slots(you can buy it ror around 200$ with 32 gb of ddr3 ram, a workstation xeon cpu and a workstation motherboard,… 20 Smol AI News news-outlet 13d ago not much happened today **OpenAI** is advancing its power-and-compute infrastructure with a **4+ GW NVIDIA** capacity commitment and an **8 GW Ohio campus** buildout through **2032**, emphasizing vertical integration across power, data centers, and chips. The model access and routing API layer is… 20 r/LocalLLaMA community 13d ago How many tokens/second output are you getting with Qwen3.8-27B? Trying to get a feel for where I stand. If you can list your relevant hardware and model used, that would be awesome. Here's mine: Model: Qwen3.8-27B-heretic-ara, Q5_K_M GGUF T/s : ~30-32 t/sec (I think, I'll verify in a bit) Hardware: 3090 GPU | 64 GBs DDR4 RAM | AMD 7950x CPU… 17 arXiv — Machine Learning research 13d ago Robust Dual-Model Collaborative Random Vector Functional Link Network arXiv:2608.13628v1 Announce Type: new Abstract: Random vector functional link (RVFL) networks are lightweight and fast neural models that offer efficient training and strong generalization through randomized hidden-layer weights and direct input-output connections. However,… 36 arXiv — Machine Learning research 13d ago The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference arXiv:2608.13756v1 Announce Type: new Abstract: Two GPU kernels implementing the same scaled INT8 GEMM interface are usually treated as interchangeable. We test that assumption: holding the checkpoint, prompts, hardware, inference engine, decoding, and quantization configuration… 25 arXiv — Machine Learning research 13d ago Non-Parametric Spatiotemporal Trajectory Prediction via State-Conditioned Transition Sampling arXiv:2608.14349v1 Announce Type: new Abstract: We present a training-free method for multi-modal trajectory prediction that achieves comparable accuracy to a 57M-parameter transformer while requiring no GPU and zero learned parameters. The method builds a transition table of… 7 arXiv — Machine Learning research 13d ago Rollplex: Cross-Phase GPU Spatial Sharing for Vision Language Model Post-Training arXiv:2608.14498v1 Announce Type: new Abstract: Vision-language models (VLMs) enable embodied agents to reason and act from visual observations and language instructions. Reinforcement learning (RL) post-training enhances these capabilities using task feedback, but current… 8 arXiv — NLP / Computation & Language research 13d ago Grounding Without Corrective Control: Truth-Tracking Profiles for Large Language Models arXiv:2608.14252v1 Announce Type: cross Abstract: Recent work suggests that some large language model representations have content or reference. Grounding can secure either without supplying live routes for correction. This paper asks what follows from that gap. An output is… 16 r/LocalLLaMA community 13d ago Qwen3.8 27B Q2 vs Q3 vs Qwen3.6 35B-A3B MoE on 12GB VRAM Did a quick local test because I wanted to see what is actually usable on my 12GB laptop GPU. I tested the newer Qwen3.8 27B dense files at Q2 and Q3, then compared them against Qwen3.6 35B-A3B MoE. Hardware: RTX 5070 Ti Laptop, 12GB VRAM Backend: llama.cpp CUDA Settings: 4k… 12 r/LocalLLaMA community 13d ago Qwen 3.8 2.4T at 288k tokens/s on Nvidia GB300 NVL72 https://developer.nvidia.com/blog/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reasoning-on-nvidia-gb300-nvl72/ 4k tokens per second per GPU of which there are 72. 350 tokens per second per user "Without additional model tuning, the model achieves a… 13 r/MachineLearning community 13d ago [Career Advice] Final-year in Physical AI / Robotics. How is the market & global hiring for freshers? [D] Hi everyone, I am heading into my final year of my BTech at a tier 1 college in India and just wrapped up a Physical AI internship at a MNC, working heavily with NVIDIA Isaac Sim and OpenFOAM. My background is fully focused on robotics and autonomy. My tech stack includes:… 35 r/LocalLLaMA community 14d ago How many people have 24gb over gpu here? I was surprised by the fact that the qwen 3.8 27b download count is about 1 million (globally). This means that even on this subreddit, very few people have used 27b. At most 50k–100k active users, and once you break down the hardware distribution, 8GB, 16GB, 24GB, 32GB cards,… 5 r/LocalLLaMA community 14d ago Need assistance in getting my 4 AMD GPU setup running on a Z10PA-U8 mainboard I'm trying to run 4x AMD RX 6900 XT and an NVIDIA GT 710 (x8) display card on an ASUS Z10PA-U8 mainboard and Ubuntu Server 24.04. But so far this doesn't work so well. I have an old GA-Z77-DS3H mainboard which has 1 PCIe x16 (Gen3), 2 PCIe x1 (Gen2) and 1 PCIe x16 (Gen2) slots.… 29 Simon Willison community 14d ago CORS Chat Tool: CORS Chat I built this today ( with GPT-5.6-Sol xhigh ) to help test Qwen 3.8 27B running in LM Studio on both my M5 MacBook Pro and an NVIDIA DGX Spark. It provides a web UI for exercising an OpenAI-Responses-compatible chat endpoint. I've tried it against LM Studio with… 36 r/MachineLearning community 15d ago Do you actually finish setting up a new project? [N] I have a bad habit of getting a new project 90% of the way there and then losing interest. Dependencies work, GPU is detected, model downloads, everything finally runs. And then somehow the actual project gets abandoned. I'm starting to think getting things working is half the… 22 Page 5 of 10 · 500 articles ← Newer Older →
