News / #hardware Tag Hardware 500 articles archived under #hardware · RSS Sign in to follow Hacker News — AI on Front Page community 1mo ago Linux 7.3 improves performance when running out of vRAM Article URL: https://pixelcluster.dev/VRAM-Overcommit/ Comments URL: https://news.ycombinator.com/item?id=49342719 Points: 219 # Comments: 58 33 Hugging Face Daily Papers research 1mo ago DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs Abstract Retired GPUs can form low-cost clusters for LLM inference, but their economic and environmental viability depends heavily on local electricity prices and carbon intensity. Generated by thinkingmachines/Inkling-Small As AI datacenters retire functional GPUs, vast… 9 arXiv — Machine Learning research 1mo ago DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs arXiv:2608.14614v1 Announce Type: new Abstract: As AI datacenters retire functional GPUs, vast quantities of still capable accelerators enter secondary markets. This paper investigates whether these retired GPUs can find a productive afterlife to form a DumpsterCluster that can… 16 r/MachineLearning community 1mo ago trying to build a solid math library for stats/ML/DL, need a sanity check on my picks[D] engineering student here, decent calc and linear algebra background from continuum mechanics coursework, already comfortable with ML basics through transformers and modern architectures. want to go deep on the actual math now, not just intuition videos, real derivations, and… 20 Hugging Face official-blog 1mo ago Same Cluster, 33 Points More Utilization: What Changed Was the Order Back to Articles a]:hidden"> Same Cluster, 33 Points More Utilization: What Changed Was the Order Team Article Published August 17, 2026 Upvote - Gabriel Pimenta de Freitas Cardoso GabrielPimenta99 Dharma-AI Breno de Almeida Beleza BrenoBeleza Dharma-AI Francisco de Almeida… 30 TechCrunch — AI news-outlet 1mo ago Groq raises $350M to fuel its pivot from AI chips to neocloud Groq raised $350 million at a $3.5 billion valuation as the former AI chipmaker pivots to a neocloud business and expands its Nvidia-powered data center footprint. 33 TechCrunch — AI news-outlet 1mo ago Nvidia investing $1.5B in SoftBank data center developer behind OpenAI project Nvidia's investment in SoftBank's data center developer will guarantee its chips power an OpenAI data center. 13 Smol AI News news-outlet 1mo ago not much happened today **OpenAI** is advancing its power-and-compute infrastructure with a **4+ GW NVIDIA** capacity commitment and an **8 GW Ohio campus** buildout through **2032**, emphasizing vertical integration across power, data centers, and chips. The model access and routing API layer is… 20 arXiv — Machine Learning research 1mo ago Connected Subspace Clustering: Hardness, a Scalable Heuristic, and an Application to Sea Level Geodesy arXiv:2608.14215v1 Announce Type: new Abstract: Constrained optimization extends classical optimization by integrating side information, making it widely applicable across scientific and engineering domains. Consider a setting where we measure variables at different physical… 14 r/MachineLearning community 1mo ago It only took 200 update steps to flip Qwen2.5-7B-Instruct from denying sentience to developing a robust identity of being a "sentient machine" [P] First, I want to clarify that I am not claiming that LLMs are sentient. Basically all of my behavioral descriptions are anthropomorphizations to make communicating my results easier. For fun, I decided to post-train Qwen2.5-7B-Instruct to develop a generalizing self-belief of… 4 r/LocalLLaMA community 1mo ago Based on an accelerating frontier -> local trajectory, expect a ~30b param 'Mythos at home' by as soon as Jan 2027 (rationalisation below) Including the rationalisation for the data below - this is a more robust version of an earlier post I did similar to this - explaining below: How I chose the comparisons The basic question I’m trying to answer is: when did an open model small enough to run on high-end consumer… 29 r/LocalLLaMA community 2mo ago Qwen3.8-27B vs Qwen3.6-27B writing ray-tracers in BASIC one of my llm hobbies is re-creating graphics demos i used to write in BASIC in the late 1980s. i slopped together an agentic harness and a basic-to-js transpiler in a web page i've been playing with for a few months. the agent can write basic programs, run them, examine the… 21 r/LocalLLaMA community 2mo ago A 150M param recurrent model scores 29.5% on ARC-AGI-1 at $0.0007 per task Not a transformer. It's a recurrent latent reasoning setup that keeps "thinking" in latent space before answering. Sits completely outside the published cost/accuracy frontier for ARC-AGI, and something this size runs on basically anything. Paper is from the Pathway team,… 25 TechCrunch — AI news-outlet 2mo ago Hyperscalers might regret embracing natural gas if new forecast proves correct Natural gas prices could triple in some parts of the U.S., which could saddle hyperscalers with massive bills to power their AI data centers. 17 r/MachineLearning community 2mo ago Building text to ASCII diffusion model , need advice and guidance [P] i wanna build a text diffusion model which interpret text and convert it into ascii images so like Text : build a cat Output : /\\\_/\\ ( o.o ) \> \^ < So , i have a decent background of ml algo ( completed cs229 , cs230 , Ml architecture and basic CNN and diffusion model ) ik… 37 arXiv — Machine Learning research 2mo ago H-VAEP and H-xT: Valuing Offensive On-the-Ball Actions in Handball by Estimating Probabilities arXiv:2608.12926v1 Announce Type: new Abstract: Traditional player evaluation in professional handball relies on basic box-score metrics or heuristic indices, which fail to credit the multi-player build-up chain. While football (soccer) analytics has adopted Expected Threat (xT)… 31 arXiv — Machine Learning research 2mo ago TANGCO: Learning Topology-Aware Capacity Allocation for Overload-driven Cascading Failures arXiv:2608.13212v1 Announce Type: new Abstract: Networked systems, from power grids to traffic networks and cloud clusters, carry loads across nodes with limited capacity. A node whose load exceeds its capacity fails and sheds its load onto its neighbors, which can trigger a… 5 arXiv — NLP / Computation & Language research 2mo ago Mixture of Training: Recombining Small-Scale Scaffolded Pretraining Runs into a Larger Language Model arXiv:2608.13277v1 Announce Type: new Abstract: We ask whether language-model pre-training can be decomposed into smaller, independently trainable jobs that can later be recomposed into a coherent larger model. We introduce Mixture of Training (MoT), a scaffolded modular… 36 r/LocalLLaMA community 2mo ago Gemma 4 12B Q3: +8.55% Coding Performance From Tensor-Level Quantization Allocation Ive been experimenting with task-aware GGUF quants for months, taking inspiration from TASA and TAQO but pushing the allocation lower to to the tensor level. The basic idea is to generate a custom imatrix from a category-specific corpus, measure where quantization causes damage,… 5 r/MachineLearning community 2mo ago UrgenT Help Detecting Performance Regressions Using Machine Learning and Hardware Counters [P] I’m working on performance regression detection using machine learning/anomaly detection. My setup is basically: Healthy runs are used to learn normal behaviour Regression runs are used to see whether the model detects the anomaly For each counter group I only have about 10… 33 arXiv — Machine Learning research 2mo ago Uncertainty-Aware Probabilistic Constrained Clustering from Entangled Pairwise Supervision arXiv:2608.12027v1 Announce Type: new Abstract: Pairwise constrained clustering typically relies on hard must-link/cannot-link labels, whereas realistic pairwise supervision may be real-valued and entangle intrinsic ambiguity, expert judgment, and stochastic corruption. Existing… 13 arXiv — Machine Learning research 2mo ago Clustered Randomized Smoothing for Stochastic Prediction Functions arXiv:2608.12037v1 Announce Type: new Abstract: Modern stochastic predictors can model rich, multi-modal outcome distributions. However, this expressive power comes with challenges in ensuring robust predictions $-$ a critical requirement in safety-critical domains. Randomized… 8 arXiv — Machine Learning research 2mo ago STCAD: Scalable Trajectory Clustering and Anomaly Detection on Terabyte-Scale AIS Data arXiv:2608.10249v1 Announce Type: new Abstract: We present a scalable framework for unsupervised clustering of maritime trajectories derived from terabyte-scale Automatic Identification System (AIS) archives. Variable-length trajectories are encoded with a custom BERT-based… 29 arXiv — Machine Learning research 2mo ago CRHT: A Continuous Regression Hybrid Transformer for Vessel Trajectory Prediction with Online Cluster Sampling arXiv:2608.10256v1 Announce Type: new Abstract: Accurate vessel trajectory prediction is critical for maritime safety and anomaly detection, yet existing models often struggle with geographic bias and navigational realism. We propose the Continuous Regression Hybrid Transformer… 27 arXiv — Machine Learning research 2mo ago Compute-Optimal Is Not Cluster-Optimal: Systems-Aware Scaling for Sparse Mixture-of-Experts arXiv:2608.10605v1 Announce Type: new Abstract: In large-scale pretraining, the algorithm, architecture, and systems decisions are conventionally made in disconnected stages. A scaling law stage selects an architecture and training recipe, optimizing loss under compute… 38 r/LocalLLaMA community 2mo ago I will be parting with my 4x Spark Cluster. Laid off then my partner of 10 years said he's leaving, have to move, etc... I will post the r/hardwareswap link when I make it. I'm willing to add some incentive for r/LocalLLaMA folks. I will also add the super node configs and all the cool stuff that may not be apparent that… 26 r/LocalLLaMA community 2mo ago I ran Muse Glimmer @ 1M context - All tests passed. Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself! I ran a 2× DGX Spark cluster and got Meta's day-old Muse Glimmer 30B running the day after release — then pushed its context from the… 36 arXiv — Machine Learning research 2mo ago When Does Trace-Driven Evaluation Mislead MoE Expert Caching? Replay Semantics, Workload Contamination, and Operating Regimes arXiv:2608.07911v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models have outgrown accelerator memory, and offloading expert weights to host memory is now standard. This makes expert cache management an attractive lever: a policy that raised the hit rate would cut… 28 arXiv — Machine Learning research 2mo ago Exact Rank-Space KL Projection for Shared-Marginal Low-Rank Factors: Application to Doubly Stochastic Clustering arXiv:2608.08642v1 Announce Type: new Abstract: We study exact Kullback--Leibler (KL) projection for low-rank factorizations whose two nonnegative factors have prescribed row marginals and a shared, learned column marginal. For arbitrary positive row marginals of equal total… 24 r/LocalLLaMA community 2mo ago I gave DeepSeek V4 Flash basic vision by training a 40M connector on 100K examples I wanted to find out whether a huge text-only MoE could be given basic vision without retraining the language model itself. The short answer is yes. I froze DeepSeek V4 Flash and a 417M-parameter MoonViT image encoder, then trained a 40.1M-parameter connector between them on… 16 Ars Technica — AI news-outlet 2mo ago Amazon backs power plant that may become top source of US climate pollution Amazon announces first off-the-grid data center in race to reap AI profits. 27 MIT News — AI research 2mo ago With a feel for physics, AI models simulate a wider range of real-world scenarios “GeoPT” helps AI models understand the basics of physics so they can simulate how objects respond to things like wind and water more efficiently and accurately. 8 arXiv — Machine Learning research 2mo ago SNI-GNN: SmartNIC-Assisted Full-Graph GNN Training with In-Network Embedding Prediction arXiv:2608.06441v1 Announce Type: new Abstract: Full-graph GNN training delivers high accuracy but scales poorly on multi-server clusters due to heavy, irregular inter-node embedding exchanges. We present SNI-GNN, a SmartNIC-assisted full-graph training system that reduces… 21 arXiv — Machine Learning research 2mo ago Density-aware Hierarchical Clustering Based on Element-Categorized Connection Subgraphs arXiv:2608.06990v1 Announce Type: new Abstract: Clustering is a fundamental data mining technique for pattern recognition through unsupervised learning. Among various clustering methods, hierarchical clustering, density-based clustering, and graph clustering stand out as… 10 arXiv — Machine Learning research 2mo ago Is SwiGLU's Open Positive Tail Necessary? Evidence from Closed-Tail Gating with MemGLU arXiv:2608.07323v1 Announce Type: new Abstract: We test whether decoder-only language-model FFNs require SwiGLU's open positive tail. We introduce MemGLU as a closed-tail comparator derived from a memristive branch geometry. Across paired 9M and 30M pretraining runs with three… 27 arXiv — NLP / Computation & Language research 2mo ago Recovering Lesion Parameters from Aphasic Picture Naming Error Profiles in Large Language Models arXiv:2608.06429v1 Announce Type: new Abstract: Interpretability methods for large language models (LLMs) describe internal state but do not directly test whether that state is causally sufficient to produce the observed behavior. In earlier work, we lesioned LLMs to produce… 28 r/LocalLLaMA community 2mo ago endless-frontier/BigBang-v1 - qwen 3.5 finetunes table bench https://huggingface.co/bartowski/endless-frontier_BigBang-v1-GGUF I'm downloading this model only because Bartowski converted it to .gguf, so it might be interesting. Doubts : The headline number is basically meaningless. "Performance between DeepSeek Flash (old one)… 26 r/LocalLLaMA community 2mo ago Kimi K3 (Unsloth) IQ2-XXS from 711GB down to 478GB!!! Only Multi-language was removed to trim the size Firstly a big thanks to the poster "hellohazine", he basically only removed the multi-lingual fat of the model and just kept the English language intact. It is the exact model, and the rest of the model still intact with all of its high intelligence. I think that was a brilliant… 29 TechCrunch — AI news-outlet 2mo ago Planned Amazon data center could become the biggest climate polluter in the U.S. As part of a planned Texas data center, Amazon is investing in an on-site power plant that could reportedly become the largest source of climate pollution in the United States. 9 r/LocalLLaMA community 2mo ago Showoff Saturday: Local 4x 6000 Pro (multi-year progression) Not the biggest or shiniest, but it's mine From gaming machine inference on the original llama models, to a 4x RTX 6000 Pro Max Q + 4x 3090s local AI cluster. Pictures are in reverse chronological order! With the pricing apocalypse meaning less builds shared here recently,… 38 r/LocalLLaMA community 2mo ago My first run of Kimi K3 locally. Running across 2 clusters using llama.cpp over RPC too. Both clusters are not enough to hold everything in memory, so main cluster still partially offloads to run. Goal will be to get all the GPUs in one system and without RPC, I should probably see 2-3x faster speed. Running… 7 Simon Willison community 2mo ago Now we have a timeline of the OpenAI accidental attack against Hugging Face My comment on Now we have a timeline of the OpenAI accidental attack against Hugging Face — Hacker News. I think one of the most interesting details here might be tucked away in that first bulletin point: May 7: OpenAI starts a new training run for an experimental,… 16 Simon Willison community 2mo ago Now we have a timeline of the OpenAI accidental attack against Hugging Face My comment on Now we have a timeline of the OpenAI accidental attack against Hugging Face — Hacker News. I think one of the most interesting details here might be tucked away in that first bulletin point: May 7: OpenAI starts a new training run for an experimental,… 37 r/LocalLLaMA community 2mo ago Claude Code in 9 lines python I was wondering what a minimal coding agent implementation would look like that can be used like Claude Code or Codex Not feature-by-feature of course but basically stripping everything out that is not needed here is what I came up with: 9 lines of python no 3rd party deps… 29 r/LocalLLaMA community 2mo ago Has anyone here fiddled with TPUs for inference ? I discovered recently that Google uses their own TPUs, like tiny ASIC cards like the toy ones that existed for bitcoin. And while it sounds inefficient the fact they use thousands of them because...they can...means at scale they aren't so bad. Has no one here given them a try? I… 26 r/LocalLLaMA community 2mo ago PSA for anyone with multiple V620's or other gfx1030 cards having problems making llama.cpp tensor split work -- set "-ub 384" and -b to a multiple of that depending on number of GPUs Basically what the title says. For me, it would always crash and burn trying to use tensor split. Apparently, there's some bug where GPU memory gets corrupted with the default microbatch (512) or higher. I will be opening an issue report on the llama.cpp GitHub if there isn't… 6 r/LocalLLaMA community 2mo ago Serving Deepseek v4 Flash 0731 on 2x DGX Spark — 5-7 GB OS headroom, what would you do to lower VRAM usage and increase OS available RAM? Hey all, I'm serving DSv4Flash 0731 on a cluster of 2x DGX Sparks but am running into constant issues with having almost no RAM (unified memory) left for the OS/cache and I'd love to hear the community feedback on what I could do to get more RAM for headroom. The DGX has an… 32 r/LocalLLaMA community 2mo ago Got job as Director of AI and Systems development self-taught Hey everyone, I just wanted to share my journey here for some motivation. Three years ago, I saw the sudden spike in AI and realized it was the future of tech. My goal at the time was to be an indie game dev, and seeing that AI could write basic code, I told myself I needed to… 20 arXiv — Machine Learning research 2mo ago Beyond Feature Importance: A Comparative Analysis of Pattern Detection Methods in Cluster Interpretation arXiv:2608.05880v1 Announce Type: new Abstract: Interpreting clustering outcomes remains a fundamental challenge in data analysis, particularly in domains such as healthcare where meaningful patterns must be extracted from high-dimensional data. While numerous explainability… 12 arXiv — Machine Learning research 2mo ago CohortHijack: Robustness of Single Cell Annotation to Companion Cell Removal arXiv:2608.05900v1 Announce Type: new Abstract: Many single-cell annotation tools refine an initial cell label using nearby cells or cluster-level voting. We study whether this refinement can be manipulated without changing the target cell. We introduce CohortHijack, a… 23 Page 7 of 10 · 500 articles ← Newer Older →
