specs

hardware: M4 Max 128GB Studio
inference engine: oMLX & lllama.cpp

insights

it still very early, so had to disable oMLX K/V caching, qwen4_exp architectureis not yet supported
+ the obvious n-grams with which the whole 4 bit quant takes ~100G, so pretty tight

nevertheless, this is the first model for the year that was able to break through 94% on my cupel benchmark

one interesting bit is Qwen 3.8 27B is obviously great, but it did not do that well, since I have coding, general knowledge and science. it did outperform most in coding, but its general knowledge lost to Gemma 31B as well as to Qwen 3.6

omlx

this is the quant I tried with oMLX, which performed better than other 4 bit quants due to the mixed quantization:

pipenetwork/Qwen3.8-Flash-Next-MLX-mixed-4_8bit

buld	perplexity
bfloat16	4.4708
mixed-4_8bit	4.5286

llama.cpp

this is a very good quant from Unsloth, it is not as strong as "MLX-mixed-4_8bit", but I could not fit a larger one from unsloth to be able to bench. You can see it on position #6 in the above leaderboard

unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS

I am working on collecting all I did for the last few months codingwise, and will add more pieces into the benchmark (hermes => pi / opencode, etc..) because models are getting too good to differentiate: I love it!