7900 XTX — two "low-thinking" Qwen 3.8 27B quants (Swift + ThinkingCap) vs the regular quant

First, do they actually produce less tokens?

Yes. Total tokens per benchmark run (4 scenarios): base quant ~66k, ThinkingCap ~49k (−26%), Swift ~45k (−33%). So the "less thinking" is real — and Swift cuts the most.

Then the cost:

and this is where it got interesting. The two quants don't trade off the same way:

• Decode: base ~48 t/s. ThinkingCap barely changes (~43). Swift drops hard (~32, −33%).
• Prefill / TTFT — the opposite of what I expected: Swift is the fastest (~614 t/s, TTFT ~1s), base in between (~530 t/s, ~1.2s), ThinkingCap the slowest (~100 t/s, TTFT 6–11s).
• Quality holds: ~84–87 on my eval, same band as base.

Full run data:

ThinkingCap finishes ~23% faster than base. The token savings win even with the slow prefill. Swift is break-even because of the slower decode speed.

Caveat: 2 runs per quant only, so single-run variance will move these. Prefill speed of ThinkingCap is oddly low. Need to do some more tests on that.

Side-by-side (thinking xhigh, Q4_K_M, all 7900 XTX) of 3 of the runs:

https://llm-bench.io/compare/runs?runs=cmufxkpkb00bj01o05h9vsu5o,cmufwold900ay01o0ae1frktu,cmufyiq3p00by01o0vwbpilew