Running Vision Qwen 3.8 27B on a 16GB Card, the config (45tks).

I am just sharing my config for Qwen 3.8 27b that fits on a 5060TI, what is cool about this is that you can even get vision! and a 85K context (I have 1.5gb of headroom for more context or a better quant)

Model: IQ3_XXS-mtp from https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

Using beellama https://github.com/Anbeeld/beellama.cpp

Config used:

[*] model = ..\llm-models\Qwen3.8-27B-GSQ-RCO-IQ3_XXS-mtp.gguf mmproj = ..\llm-models\mmproj-Qwen3.8-27B-BF16.gguf image-min-tokens = 256 gpu-layers = 99 ctx-size = 85000 no-host = true direct-io = true threads = 8 batch-size = 2048 ubatch-size = 512 fit = off ctx-checkpoints = 0 spec-type = draft-mtp spec-draft-n-max = 2 cache-type-k = kvarn4 cache-type-v = kvarn4 kv-tail-tokens = 256 

I managed to get 45tks on decode and around 300 on prefill

Yes it is using kvarn4, but it is not that bad, check:
https://anbeeld.com/articles/kvarn-kv-cache-implementation-and-benchmarks

I also know that you could move the mmproj to cpu to to gain more vram.

Would love to hear other configurations to find the sweetspot for 16GB vram cards!