Turns out you can use very very old hardware to serve relatively large models with a magnitude reduction in throughput</p>\n","updatedAt":"2026-08-18T06:05:47.626Z","author":{"_id":"6475c2794766357252e69e9f","avatarUrl":"/avatars/757ed789423113369868e972f21ce559.svg","fullname":"i","name":"iliashum","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":8,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9729970693588257},"editors":["iliashum"],"editorAvatarUrls":["/avatars/757ed789423113369868e972f21ce559.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.14614","authors":[{"_id":"6a83f5ef675db694db8cd61f","name":"Zeyu Cao","hidden":false},{"_id":"6a83f5ef675db694db8cd620","name":"Xuan Guo","hidden":false},{"_id":"6a83f5ef675db694db8cd621","name":"Cheng Zhang","hidden":false},{"_id":"6a83f5ef675db694db8cd622","name":"Cheuk Hang Lau","hidden":false},{"_id":"6a83f5ef675db694db8cd623","name":"Ilia Shumailov","hidden":false},{"_id":"6a83f5ef675db694db8cd624","name":"Yiren Zhao","hidden":false}],"publishedAt":"2026-07-10T00:00:00.000Z","submittedOnDailyAt":"2026-08-18T00:00:00.000Z","title":"DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs","submittedOnDailyBy":{"_id":"6475c2794766357252e69e9f","avatarUrl":"/avatars/757ed789423113369868e972f21ce559.svg","isPro":false,"fullname":"i","user":"iliashum","type":"user","name":"iliashum"},"summary":"As AI datacenters retire functional GPUs, vast quantities of still capable accelerators enter secondary markets. This paper investigates whether these retired GPUs can find a productive afterlife to form a DumpsterCluster that can serve modern LLM inference, and under what conditions such repurposing is economically viable and environmentally sustainable. We physically built a 128-GPU DumpsterCluster from scratch using only second-hand components and ran it for one year. At current market prices (\\22K for the DumpsterCluster vs. 600K for an 8-GPU B200 system), the economic advantages are substantial. Through pipeline-parallel optimizations, our V100 based DumpsterCluster achieves competitive LLaMA-70B throughput, validating production viability. However, our deployment reveals critical context dependencies. Older GPUs consume significantly more energy per token, making total cost of ownership favorable only in regions with inexpensive electricity. Under grid-average carbon intensity, second-hand systems can produce approximately 4x higher total carbon emissions per token for 8B models, and over 40x for 70B models, compared to current-generation hardware. These findings show that GPU afterlife is not universally sustainable - hardware repurposing must be strategically coupled with low carbon energy sources. When deployed in regions with favourable energy economics and clean electricity, second-hand GPUs offer a viable pathway for expanding AI capacity while advancing affordability, energy security, and environmental responsibility.","upvotes":1,"discussionId":"6a83f5ef675db694db8cd625","ai_summary":"Retired GPUs can form low-cost clusters for LLM inference, but their economic and environmental viability depends heavily on local electricity prices and carbon intensity.","ai_keywords":["DumpsterCluster","pipeline-parallel optimizations","LLaMA-70B","total cost of ownership","carbon intensity","second-hand GPUs"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"627bbc28fbab61b048eba8b6","name":"Oxford","fullname":"University of Oxford","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/68e396f2b5bb631e9b2fac9a/u0ey2LfYu6uG6iu8m_kH7.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"627bbc28fbab61b048eba8b6","name":"Oxford","fullname":"University of Oxford","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/68e396f2b5bb631e9b2fac9a/u0ey2LfYu6uG6iu8m_kH7.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.14614.md","query":{}}">
DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs
Published on Jul 10
· Submitted by i on Aug 18 Abstract
Retired GPUs can form low-cost clusters for LLM inference, but their economic and environmental viability depends heavily on local electricity prices and carbon intensity.
As AI datacenters retire functional GPUs, vast quantities of still capable accelerators enter secondary markets. This paper investigates whether these retired GPUs can find a productive afterlife to form a DumpsterCluster that can serve modern LLM inference, and under what conditions such repurposing is economically viable and environmentally sustainable. We physically built a 128-GPU DumpsterCluster from scratch using only second-hand components and ran it for one year. At current market prices (\22K for the DumpsterCluster vs. 600K for an 8-GPU B200 system), the economic advantages are substantial. Through pipeline-parallel optimizations, our V100 based DumpsterCluster achieves competitive LLaMA-70B throughput, validating production viability. However, our deployment reveals critical context dependencies. Older GPUs consume significantly more energy per token, making total cost of ownership favorable only in regions with inexpensive electricity. Under grid-average carbon intensity, second-hand systems can produce approximately 4x higher total carbon emissions per token for 8B models, and over 40x for 70B models, compared to current-generation hardware. These findings show that GPU afterlife is not universally sustainable - hardware repurposing must be strategically coupled with low carbon energy sources. When deployed in regions with favourable energy economics and clean electricity, second-hand GPUs offer a viable pathway for expanding AI capacity while advancing affordability, energy security, and environmental responsibility.
Community
Turns out you can use very very old hardware to serve relatively large models with a magnitude reduction in throughput
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.14614 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.14614 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.14614 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.