Buying a V100/older NVIDIA GPU? Run this to check for older memory issues
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
| | I recently bought a 32GB V100 off eBay and thought I was all set with rudimentary tests showing zero issues. But then I started seeing weird VRAM-related errors in llama.cpp. I sic'ed Claude on it, to find that ECC was disabled and that can hide small RAM issues. I believe the aggregate ECC error counter can also be reset. However, if ECC is enabled and RAM starts failing, a record of that is stored in the InfoROM on the card with timestamps. This script can be run to see if you've got any retired pages, and on my card it showed retired pages going back to 2019. Another V100 (16GB) I have has zero, as expected. Unfortunately, I didn't learn all this until after my 30 days was up, so now I'm stuck with a $900 paperweight. I'll run this on the next V100 I order immediately after installing it to ensure I'm not getting stuck with someone's e-waste. https://gist.github.com/samteezy/a788dcf430deb448ae48bbe17c369241 [link] [comments] |
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.