Llama.cpp DSpark PC Tree Fork (up to 3%-29.5% faster!)
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
Hello gang, I made an implementation of DSpark PC Tree (Parent conditioned drafting tree).
This is an implementation of this research paper: https://arxiv.org/abs/2608.02123
Unaffiliated, just found it and implemented it.
And I have to preface: This is just a first shot, I have no feedback from anyone yet!
These are some stats im getting with Qwen 3.0:
GPU: SM120 (RTX5090) llama-bench combined Configuration tok/s vs plain vs DSpark n3 Acceptance ━━━━━━━━━━━━━━━ ━━━━━━━━ ━━━━━━━━━━ ━━━━━━━━━━━━━━ ━━━━━━━━━━━━ Plain 94.27 1.00x -39.43% n/a ─────────────── ──────── ────────── ────────────── ──────────── DSpark n2 138.83 1.47x -10.80% 55.66% ─────────────── ──────── ────────── ────────────── ──────────── DSpark n3 155.64 1.65x baseline 47.19% ─────────────── ──────── ────────── ────────────── ──────────── PCTree k2/n8 152.29 1.62x -2.15% 58.83% ─────────────── ──────── ────────── ────────────── ──────────── PCTree k3/n16 159.00 1.69x +2.16% 67.87% ─────────────── ──────── ────────── ────────────── ──────────── PCTree k4/n22 157.99 1.68x +1.51% 72.16% k3/n16 beat linear DSpark in 9 of 11 categories. Its largest gain was summarization at +6.56%. Only RAG (-0.57%) and multilingual (-1.35%) slightly favored linear. k4/n22 accepted more but drafts but could not repay the larger verifier batch. Benchmark conditions: SPEED-Bench qualitative, all 11 categories, four samples per category, 44/44 completed with zero failures, 512 output tokens, temperature 0, 8K context, one slot, BF16 target and draft. CPU-only llama-bench, 12 threads, three repetitions: Model pp512 tg128 ━━━━━━━ ━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━━ BF16 76.59 +/- 0.33 tok/s 3.82 +/- 0.02 tok/s ─────── ────────────────────── ───────────────────── Q8_0 76.07 +/- 0.79 tok/s 7.12 +/- 0.01 tok/s The PCTree launch parameters are: --spec-type draft-dspark --spec-draft-n-max 3 --spec-dspark-pctree --spec-dspark-pctree-k 3 --spec-dspark-pctree-n 16 Example complete command: llama-server.exe ` -m Qwen3-8B-BF16.gguf ` -md dspark-Qwen3-8B-BF16.gguf ` --spec-type draft-dspark ` --spec-draft-n-max 3 ` --spec-dspark-pctree ` --spec-dspark-pctree-k 3 ` --spec-dspark-pctree-n 16 ` -ngl all ` -ngld all ` -c 8192 ` --parallel 1 ` --temp 0 ` --jinja ` --host 127.0.0.1 ` --port 8080 It would be cool if you could test it out with some models and tell me if it works or not.
kinda hoped someone on llamacpp would look at it, but seemingly no one wanted to :P
I tested Qwen 3.8 27B Q4, but that was sadly worse with k2-4.
Im interested to see in how DS Flash would perform for example.
You can read a bit more below in the issue I created.
Repo: https://github.com/getraid/llama.cpp-dspark-pctree
Related research issue: https://github.com/ggml-org/llama.cpp/issues/27499
[link] [comments]
More from r/LocalLLaMA
-
Uncensored Multi-Model Releases, LongCat-Flash-Lite-Sparse with MTPs and LSAs, Qwen3.8-27B with MTPs, Qwen3.5-122B-A10B with MTPs, Qwen3-Coder-Next and Laguna-S2.1 with Vision, All Available in GGUF Format! Bonus: Links to my llama.cpp Fork for LongCat-Flash-Lite Support and…
Aug 30
-
Got MiniMax H3 video generation running in TensorSharp
Aug 30
-
Qwen3.8-Flash-Next NVFP4 2xDGX Spark config: 50t/s decode, 2,900t/s prefill
Aug 30
-
Don't Sleep on EXL3 Quants
Aug 30
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.