Qwen3.8-27B vs Frontier vs Others

Here're results from my final test of coding, coding (panda/numpy), data analysis, data-driven decision-making, and local RAG domains I interact with most often.

This test, which is refined from a 500+ pack, now provides a differentiating signal between models after having weeded out tests all models consistently pass.

It’s a specific test developed based on my workflow and not at all an endorsement or damning of the models for all types of use cases. Aim is to identify models for heavy spreadsheet, rounding precision, array, coding type work that I can then trust the models to make decisions and act on. Verdict is review of any work I do locally has to be reviewed by Claude or gpt.

When Qwen4 or others get released, if they do better on this, gives me real good feel for my use case. Also good canary to when I can go fully local and invest in pricey gear.

https://preview.redd.it/5v05e81vu2vh1.png?width=2970&format=png&auto=webp&s=5bdaf24fab819c3f39bc385120988c3d99f8830c

• Models on OpenRouter sometimes disclose their quantization and sometimes not.
  • or-qwen-qwen3.8-27b is stock at bf16
  • or-qwen-qwen3.8-flash-next didn't disclose its quantization. In this test, it is the model that gets truncated the most (max token per test 16,384). Even after raising the max token for it from 16k to 32k, the number of truncations due to overthink didn't disappear entirely.
• iq3 GSQ-RCO is decent performer even placed among Q4s
• mradermacher's fine-tunes continue to standout for my type of work.
• Muse Glimmer, Qwen3.6-35B-A3B etc. included for comparison.

https://preview.redd.it/3i7z3hbyw2vh1.png?width=2550&format=png&auto=webp&s=a3b84092baf491c62c8bd8e6ca8c38a1205635dd

https://preview.redd.it/tqg1u9g3w2vh1.png?width=2924&format=png&auto=webp&s=21f41501b2cb7dce4bb11d9d29f60250e895fef8