r/LocalLLaMA · · 2 min read

What should I test when comparing Qwen3.6-27b quants for real world effects that humans could reason about?

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

I tried to find some good comparisons on how different quants of Qwen3.6-27b perform in different scenarios, but I failed to find good information on what kinds of real world effects there are to running different quants like Q4_K_M, UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL and UD-Q8_K_XL.

My main motivation would be to understand how much performance and context should someone sacrifice to get a bigger model quant (in different scenarios) if they have a consumer grade desktop with two GPUs that have 32GB of total VRAM.

I am targeting this build specifically, as I believe this is the only configuration you can currently buy reliably off the shelf with a reasonable budget, and still packs a decent punch for local LLM use with Qwen3.6-27b.

If I wanted to run the tests myself, what would be actually meaningful tests to run? I have personally been mostly using vanilla Pi to do some coding and more complex processing tasks with the models, but I would also be interested in other use cases/harnesses, and could run tests for them also. Preferably I would be running the tests with llama.cpp, as I have the most experience setting that up on my Ubuntu.

So what do you think would be the things these tests should look for and measure? Are there ready made tests I could easily run, which offer reasonable correlation to something we humans could easily reason with when choosing model quants?

Do you also think I should vary other things than just the base model quant with my tests, or do you think it would suffice to run all tests with just q8_0 kv and one of the two thinking parameter variants the Qwen3.6-27b model card refers to (general tasks & precise coding) depending on the test?

Also if these tests already exist and I was just too dumb to find them, I would appreciate it if you sent me a link ;)

submitted by /u/panamory
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA