DeepSeek V4 Flash 0731 on Strix Halo: draft model, n_max sweep, and a launch line that actually helps

I know the front page is completely dominated by the Qwen3.8 27B. I know it's a fantastic model and everyone is rightfully discussing it, but I wanted to shift gears and talk about another model I’ve been experimenting with and got some solid results from: DeepSeek-V4-Flash-0731.

I just finished a week of running numbers on a Strix Halo box and figured I'd dump them here before I forget what I actually measured.

Hardware is a Strix Halo mini PC, Ubuntu, 128 GB unified memory.

Target is Unsloth's UD-IQ3_XXS with Q6 attention (~96 GB). At 264k context, no context quantization, the process sits at about 114 GB.

At 64k I see 200–220 tok/s prefill and 20–22 tok/s decode with no draft. That's the baseline I used for everything below: 20.48 tok/s.

I compared two DSpark drafters, Q8_0 (10.15 GB) and a Q2_K_S I quantized myself (6.45 GB), over n_max 2–7, 7 prompt types, 5 runs each.

Same sampling the whole way: temp 0.9, top_p 0.95, min_p 0.01. Thinking on (reasoning_effort=low for the sweep so it wouldn't take a week). ngram-mod was off during the sweep so draft_n is only the drafter.

Short version: n_max=3 is the peak. Average over both drafts is 28.5 tok/s, 1.39× the no-draft baseline. Q2 and Q8 are a wash. Every n_max, every category, they land inside 1–3% of each other. So the 6.45 GB file is the one I'd actually keep.

One caveat on that 28.5 average: the suite is stacked with tasks that speculative decoding loves (repeat, math, code), and the prompts only used up to ~32k of context. Don't treat it as what you'll see on a mixed chat/coding day with a fuller window. Realistically I'd budget 22–28 tok/s.

Best n_max per category (avg vs 20.48 tok/s):

• code — n_max 3 — 28.99 t/s — 1.42×
• json — n_max 3 — 28.59 — 1.40×
• math — n_max 3 — 31.22 — 1.52×
• dialog — n_max 3 — 25.48 — 1.24×
• translate — n_max 2 — 25.09 — 1.22×
• prose — n_max 2 — 23.38 — 1.14×
• repeat — n_max 5 — 40.17 — 1.96×

n_max 5–7 only pays off on the high-acceptance stuff (repeat, some math). On prose / translate / dialog the extra draft tokens just get rejected and you go backwards. Overall table if you want it:

n_max 2: 27.12 t/s, 1.32×, accept 0.69 n_max 3: 28.50, 1.39×, 0.60 n_max 4: 27.73, 1.35×, 0.52 n_max 5–7: ~26.4–26.5, 1.29×, accept ~0.45

I run this day to day at 128k with ngram-mod stacked on the drafter and thinking set to max. That's not what the sweep used (sweep was a 64k window, ~32k of it actually filled, draft-only, thinking=low), so don't treat the 28.5 as a promise for this exact line. The n_max=3 result is what I kept; 22–28 tok/s is what I'd quote for mixed use.

llama-server \ -m DeepSeek-V4-Flash-0731-UD-IQ3_XXS-q6kattn.gguf \ -a DeepSeek-V4-Flash-0731-UD-IQ3_XXS \ --no-ui -ngl 999 -c 131072 --jinja -fa 1 --port 9989 \ --no-mmap --no-warmup -np 1 \ --temp 0.9 --top-p 0.95 --min-p 0.01 --host 0.0.0.0 \ -ngld 999 -fit off \ -md DeepSeek-V4-Flash-0731-DSpark-Drafter-Q2_K_S-dflash.gguf \ --spec-type ngram-mod,draft-dspark --spec-draft-n-max 3 \ -ub 1024 --cache-ram 2048 \ --chat-template-kwargs '{"reasoning_effort":"max"}' 

Fork llama.cpp is strix-halo-llamacpp (FA + MoE-prefill fixes, bundled Mesa, Vulkan/HIP).

GGUF I actually load: DeepSeek-V4-Flash-0731-StrixHalo-Verified-GGUF.

Drafter: Lynxpda/DeepSeek-V4-Flash-0731-DSpark-Drafter-Q2_K_S-GGUF.

Charts from the sweep (baseline drawn at 20.48):

dashboard

speedup_grouped_bars

If you're on Strix Halo and already sitting on Flash 0731, --spec-draft-n-max 3 and the Q2 drafter is what I'd start with. Happy to answer questions about the sweep if something looks off.

upd: corrected errors.