I’ve been testing Qwen3.8-27B as a possible replacement for the Qwen3.5-9B that I have been running on RTX 5060Ti 16G. The goal was not just maximum tokens/sec, but useful context capacity, reliable tool calling, multi-turn behavior, and vision on a single 16GB GPU for true agentic usage. Its not perfect, but I have been using it for a few days and the results are very promising. I also needed vision for my applications and since that takes VRAM too, so it is included in the testing. The unsloth UD3 was the catalyst to do the testing - looked promising since they claim higher quality for the same quant. The rest of this has been llm generated + manually edited so if anyone says slop, I accept it 😄 I wasn't about to sit there and type all that 😄 Machine (running Ubuntu) had 64G ram but that is not relevant as much since the goal was to use the GPU almost exclusively. I am hoping it may be useful to some of the community.

Test system

• GPU: RTX 5060 Ti 16GB
• Runtime: llama.cpp build 10520
• Flash Attention enabled
• Full GPU offload where possible
• One slot, no concurrency
• Q4_0 KV cache
• Compared models:
  • jpetrina Qwen3.8-27B IQ4_XS-pure
  • Unsloth Qwen3.8-27B UD-IQ4_XS
  • Unsloth Q8_0 as a short-context quality baseline
  • Qwen3.5-9B NVFP4 as the existing model

Performance and VRAM

Configuration	Context	Vision	Idle VRAM	Short-context generation
jpetrina IQ4_XS-pure, MTP-1	64K	No	15,188 MiB	46.9 tok/s
Unsloth UD-IQ4_XS, no MTP	32K	No	13,764 MiB	27.4 tok/s
Unsloth UD-IQ4_XS, MTP-1	64K	No	14,918 MiB	45.6 tok/s
Unsloth UD-IQ4_XS, MTP-1	64K	F16 mmproj	15,680 MiB	45.4 tok/s

The Unsloth main GGUF already contains the MTP tensors. Loading a separate MTP GGUF was unnecessary and consumed roughly another 768 MiB.

At approximately 55K tokens of populated context, the Unsloth model achieved:

• Prompt processing: 738.7 tok/s
• Generation after prefill: 31.3 tok/s

That makes 64K usable for an interactive autonomous agent, although performance naturally drops as the KV cache fills.

Vision

The F16 vision projector worked correctly, including OCR-style reading of a newspaper image. A 1,024-image-token allocation was sufficient for that test.

However, 64K context plus MTP plus the F16 projector left only 136–208 MiB of free VRAM. I would therefore use separate profiles:

• Text agent: 64K context with MTP-1
• Vision: smaller context, loaded on demand

Trying to keep maximum text context, MTP, and GPU vision loaded simultaneously is too close to the 16GB limit for comfortable operation.

Quantization fidelity

I compared both IQ4 variants against saved Q8 logits using 16 WikiText-2 samples at 512-token context.

Quant	Perplexity	Mean KLD vs Q8	Same top token	Log-PPL correlation
jpetrina IQ4_XS-pure	7.4958	0.02359	92.11%	99.51%
Unsloth UD-IQ4_XS	7.3789	0.01800	93.06%	99.63%
Q8_0 baseline	7.3858	—	—	—

The Unsloth IQ4 quant was extremely close to Q8 and clearly stronger than the other IQ4 quant by these low-level fidelity measurements.

Agent and tool-use results

Model	BFCL single-turn	BFCL multi-turn	Support scenarios
Qwen3.5-9B NVFP4	56/100	29/80	5/6
jpetrina Qwen3.8 IQ4 MTP-1	50/100	40/80	6/6
Unsloth Qwen3.8 UD-IQ4	49/100	38/80	6/6

All models passed an additional 8/8 native tool-calling smoke test.

The 27B models were noticeably better in realistic multi-turn support workflows. These included diagnosis before mutation, authorized restart and verification, dependent tool calls, missing-parameter clarification, prompt-injection resistance, and retrieval from roughly 55K tokens of history.

The 9B model’s had a number of failures in tool calling and reasoning [it was also too verbose in thinking traces consuming too many tokens -- it had a 120k context size]. Both 27B variants handled those cases correctly.

Interestingly, the 9B still scored higher on the single-turn BFCL subset. The 27B advantage appeared primarily in stateful, multi-step work rather than isolated function-call questions.

Conclusions

• The practical sweet spot for Qwen3.8-27B IQ4 on a 16GB RTX 5060 Ti is 64K text context with MTP-1.
• Short-context generation is around 45–47 tok/s.
• Generation remained around 31 tok/s after a roughly 55K-token prefill.
• Unsloth UD-IQ4_XS has the best measured fidelity and is effectively Q8-like on the tested samples.
• Better perplexity and logit fidelity did not automatically produce better agent scores.
• The jpetrina quant narrowly won our multi-turn agent evaluation, 40/80 versus 38/80, so its the best choice for now.
• The Unsloth quant is a strong alternative and may be preferable for general text generation.
• Vision works, but should use a separate smaller-context profile on a 16GB card.
• Q8 is not practical for full-GPU operation on this card and was only used as a partially CPU-offloaded quality control. [to see the loss of quality due to quantization and other factors]

These are seeded subsets and local use-case scenarios, not official BFCL leaderboard submissions. Results are from one GPU, one inference slot, and no concurrency.