100$ worth of gpu runs qwen 3.8 27b at 7.39 t/s
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
| Qwen 27b Q3_K_M 2x rx 580 8gb (~50$ each in my country, edge cases 60$ per gpu) gives us 16gb vram We used it on an old already existing ddr3 motherboard with 2 gpu slots(you can buy it ror around 200$ with 32 gb of ddr3 ram, a workstation xeon cpu and a workstation motherboard, used) Its not the best option, but it makes running this model possible for many people, its even cheaper than system ram Limitations: very low processing speed(only 14t/s) means an mtp model would be a loss, and high input tokens would be a painful experiance Not recommanded if you care about ease of life, very recommanded if you need something cheap to work no matter the compromise [link] [comments] |
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.