r/LocalLLaMA · · 1 min read

Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~20-30 tok/sec decode & 300-90k tok/sec prefill on 12GB VRAM + 32GB RAM + NVME

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

I got Qwen3.8-Flash-Next-GSQ-RCO-Abliterated running at IQ3_S with just 12GB VRAM and 32GB system RAM, achieving 20-30 tok/sec decode (Q2 achieves 39-45 tok/s) & 300 to ~90 thousand tok/sec prefill @ 131k context, on a custom fork of Strata.

This fork has tonnes of architectural changes, all are very experimental and will probably break. But the performance makes up for it. This feels like local Opus in some regards, on sub 2k in compute.

https://github.com/bodhi37/Qwen3.8-Flash-12GBRAM-32GBVRAM-SSD-Recipe
https://github.com/bodhi37/strata

submitted by /u/bodhi371
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA