yall are sleeping on qwen 3.8 27b q2 + q2 dflash + q5 kv

ok bit more context: it's actually a

QAT Q2 for Qwen 3.8 27 B: https://huggingface.co/sdkyuan/qwen3.8-27B-qat-q2_0-gguf

QAT Q2 for DFlash model: https://huggingface.co/HermiHg/Qwen3.8-27B-DFlash2-Q2_K_S-MIX-GGUF

Q5 KV seems to cause 0 problems for me; I've used it up to 200K tokens of context.

Total RAM usage is liek 13-14 ish gb and I've seen very little degradation with the QAT Q2 3.8 27B

Its just crazy that these days with a 12gb card (fits if drop down to 100K context) you can run a model thats more capable than sonnet 4.6, genuinely crazy stuff