RTX Pro 4500 vs. A100 (x4) - purchase being made today
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
Need some help from the community on this server build: RTX Pro 4500 x4 ($3,299) or used 40GB A100 x4 off eBay (around $4,000/ea).
The RTX Pro 4500 means I have to go the route of getting a PCIe switch, inviting a fair amount of complexity.
A100 means I just need to find NVLink bridge adapters and hope I’m not buying a GPU from eBay that’s on its last leg.
Those are my frantic thoughts after letting four $10k RTX 6000 Pro deals expire in my cart last week. Any feedback is appreciated. What am I missing here? What am I not seeing?
EDIT: Other than the fear of waking up and finding out all of these cards cost 10x, i'm in a hurry because I originally purchased x4 Intel B60 open box , and they have to be returned by today. I gave it my best shot, but the scaling an Intel cluster isn't worth the headache.
[link] [comments]
More from r/LocalLLaMA
-
Are there any interesting architectural innovations that we seem to be on the verge of for LLM models or AI models that might be a big deal? (Excluding maybe N-gram, since everyone is already well aware of that one)
Aug 30
-
Demo of local document extraction (52 pages) using Arctic Embed and Bonsai 8B on an Iphone 16 (KernelAI app)
Aug 30
-
Will apple still release devices with mobile HbM in 2027 ?
Aug 30
-
NVIDIA® DGX Station™ Delivering Data-Center-Class Performance from the Desktop
Aug 30
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.