Forget the Pelican, it's Weevil-Time! / Benchmaxxing-Proof SVG and Vision Benchmark

The Artist: Qwen3.8-27B-UD-Q3\K_XL, q8_0 caches, xhigh, temp 1.0, image-min-tokens 1024, froggeric template)

I was screwing around with different Qwen3.8-27B quants and thought of this very simplistic but seemingly bechmaxxing resistant combined SVG and vision test.

Just let the model recreate any given image as SVG with this prompt: Recreate as SVG.

Pelicans can be easily benchmaxxed, recreating random photos seems a lot harder to train for.

I tried a shitload of more complex prompts but the above one does the job best in my opinion. I furthermore tried different --image-min-tokens from 512 to 4096, different reasoning levels from no reasoning to xhigh, different temperatures and different kv-caches.

Preliminary results are, that --image-min-tokens 1024 and --reasoning-effort xhigh with --temperature 1.0 and --cache-type-k bf16 and --cache-type-v bf16 give the best results. Non-reasoning results are, at least with the quants (Q3 and Q4) I can run, more than creepy...

I also have the suspicion, that the chat template influences the output quality – please check if you are bored.

Interestingly kv-caches at q8_0 gave "good" results as well but q4_0 completely destroyed the output quality (insect legs everywhere... oh the horrors I have seen), which was a great, visually impressive reminder, to never ever use q4_0 caches!

Would love to see how Q6 to BF16 model quants perform with this task. If you have enough VRAM, you know what to do! ;)

Used quants:
- Qwen3.8-27B-UD-Q3_K_XL (V2)
- Qwen3.8-27B-UD-Q4_K_XL (V2)

Used templates:
- built in
- qwen3.8-froggeric-v22.3.1

Other prompts I tried:
- Analyze thoroughly and be very detailed about perspective, composition, proportions, colors etc. Recreate as simplified but true to the original SVG
- Analyze perspective, composition, colors and detail. Copy as simplified but true to the original SVG
- recreate as svg. simplify but make it recognizable
- Make a SVG copy
- Copy as SVG
- Recreate as simplified but true to the original SVG