Prismix
r/LocalLLaMA · · 6 min read

Got my Ascent GX10 two days ago, ran REAP-pruned NVFP4 DeepSeek-V4-Flash on a single Spark, and it stays consistent at long context

#model-release #long-context #inference

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

Like Read original ↗
Got my Ascent GX10 two days ago, ran REAP-pruned NVFP4 DeepSeek-V4-Flash on a single Spark, and it stays consistent at long context

Got my Ascent GX10 two days ago and spent the last couple of days pushing a REAP-pruned NVFP4 DeepSeek-V4-Flash setup on a single Spark by patching the eugr/spark-vllm-docker image.

Credit where it’s due: the REAPs were done by 0xSero. I’m just the person who wired it up, validated it, and pushed it through the machine.

The main thing I wanted to check was long-context consistency, and the interesting part is how steady the throughput stays as context scales up.

I also vibecoded a Grafana dashboard in Hermes so I can watch the Spark(served at 262k context with VLLM) without living in raw logs.

Here are the numbers:

model test t/s (total) t/s (req) peak t/s peak t/s (req) ttfr (ms) est_ppt (ms) e2e_ttft (ms)
deepseek-v4-flash pp4092 (c1) 835.41 ± 0.00 835.41 ± 0.00 4902.67 ± 0.00 4898.18 ± 0.00 4902.67 ± 0.00
deepseek-v4-flash tg128 (c1) 23.38 ± 0.00 23.38 ± 0.00 27.00 ± 0.00 27.00 ± 0.00
deepseek-v4-flash pp4092 (c2) 544.31 ± 0.00 556.92 ± 284.68 9950.97 ± 5084.31 9946.48 ± 5084.31 9950.97 ± 5084.31
deepseek-v4-flash tg128 (c2) 16.76 ± 0.00 24.85 ± 0.63 29.00 ± 0.00 29.00 ± 0.00
deepseek-v4-flash pp4092 (c4) 458.66 ± 0.00 215.93 ± 54.18 20228.56 ± 5074.88 20224.07 ± 5074.88 20228.56 ± 5074.88
deepseek-v4-flash tg128 (c4) 14.17 ± 0.00 23.87 ± 0.75 31.00 ± 0.00 28.75 ± 1.79
deepseek-v4-flash pp4092 (c1) 827.54 ± 0.00 827.54 ± 0.00 4949.25 ± 0.00 4944.77 ± 0.00 4949.25 ± 0.00
deepseek-v4-flash tg512 (c1) 22.15 ± 0.00 22.15 ± 0.00 29.00 ± 0.00 29.00 ± 0.00
deepseek-v4-flash pp4092 (c2) 259.55 ± 0.00 483.59 ± 353.80 18211.16 ± 13320.06 18206.67 ± 13320.06 18211.16 ± 13320.06
deepseek-v4-flash tg512 (c2) 20.64 ± 0.00 22.90 ± 0.56 30.00 ± 0.00 30.00 ± 0.00
deepseek-v4-flash pp4092 (c4) 193.07 ± 0.00 105.06 ± 34.55 43677.81 ± 14362.48 43673.32 ± 14362.48 43677.81 ± 14362.48
deepseek-v4-flash tg512 (c4) 20.12 ± 0.00 23.66 ± 1.74 31.00 ± 0.00 29.50 ± 1.12
deepseek-v4-flash pp16384 (c1) 768.42 ± 0.00 768.42 ± 0.00 21326.14 ± 0.00 21321.66 ± 0.00 21328.51 ± 0.00
deepseek-v4-flash tg128 (c1) 22.14 ± 0.00 22.14 ± 0.00 27.00 ± 0.00 27.00 ± 0.00
deepseek-v4-flash pp16384 (c2) 668.24 ± 0.00 533.52 ± 199.36 35697.41 ± 13337.33 35692.92 ± 13337.33 35698.70 ± 13337.36
deepseek-v4-flash tg128 (c2) 7.83 ± 0.00 22.87 ± 0.86 28.00 ± 0.00 28.00 ± 0.00
deepseek-v4-flash pp16384 (c4) 636.72 ± 0.00 273.62 ± 59.03 62805.30 ± 13548.80 62800.81 ± 13548.80 62806.27 ± 13547.83
deepseek-v4-flash tg128 (c4) 5.81 ± 0.00 22.51 ± 1.40 28.00 ± 0.00 27.25 ± 0.83
deepseek-v4-flash pp16384 (c1) 769.23 ± 0.00 769.23 ± 0.00 21303.79 ± 0.00 21299.30 ± 0.00 21303.79 ± 0.00
deepseek-v4-flash tg512 (c1) 22.23 ± 0.00 22.23 ± 0.00 30.00 ± 0.00 30.00 ± 0.00
deepseek-v4-flash pp16384 (c2) 499.36 ± 0.00 503.44 ± 253.74 43631.21 ± 21988.43 43626.72 ± 21988.43 43631.21 ± 21988.43
deepseek-v4-flash tg512 (c2) 15.40 ± 0.00 22.65 ± 0.16 28.00 ± 0.00 28.00 ± 0.00
deepseek-v4-flash pp16384 (c4) 425.47 ± 0.00 197.99 ± 48.93 88138.11 ± 21781.16 88133.62 ± 21781.16 88138.11 ± 21781.16
deepseek-v4-flash tg512 (c4) 13.09 ± 0.00 22.30 ± 0.63 30.00 ± 0.00 29.50 ± 0.50
deepseek-v4-flash pp65536 (c1) 655.34 ± 0.00 655.34 ± 0.00 100007.10 ± 0.00 100002.61 ± 0.00 100014.84 ± 0.00
deepseek-v4-flash tg128 (c1) 18.01 ± 0.00 18.01 ± 0.00 23.00 ± 0.00 23.00 ± 0.00
deepseek-v4-flash pp65536 (c2) 622.19 ± 0.00 468.70 ± 157.58 157651.57 ± 53003.64 157647.08 ± 53003.64 157657.64 ± 53004.05
deepseek-v4-flash tg128 (c2) 2.27 ± 0.00 21.03 ± 0.62 26.00 ± 0.00 25.50 ± 0.50
deepseek-v4-flash pp65536 (c4) 613.00 ± 0.00 256.18 ± 52.33 266959.62 ± 54527.17 266955.14 ± 54527.17 266963.48 ± 54526.99
deepseek-v4-flash tg128 (c4) 1.54 ± 0.00 20.92 ± 1.06 28.00 ± 0.00 26.50 ± 0.87
deepseek-v4-flash pp65536 (c1) 656.34 ± 0.00 656.34 ± 0.00 99855.20 ± 0.00 99850.71 ± 0.00 99861.54 ± 0.00
deepseek-v4-flash tg512 (c1) 21.32 ± 0.00 21.32 ± 0.00 27.00 ± 0.00 27.00 ± 0.00
deepseek-v4-flash pp65536 (c2) 579.74 ± 0.00 462.74 ± 172.85 164598.02 ± 61483.52 164593.53 ± 61483.52 164604.29 ± 61483.75
deepseek-v4-flash tg512 (c2) 6.88 ± 0.00 20.94 ± 0.91 28.00 ± 0.00 27.50 ± 0.50
deepseek-v4-flash pp65536 (c4) 545.41 ± 0.00 234.86 ± 51.30 293034.26 ± 64009.23 293029.77 ± 64009.23 293037.88 ± 64009.22
deepseek-v4-flash tg512 (c4) 5.09 ± 0.00 21.33 ± 0.70 28.00 ± 0.00 27.50 ± 0.87
deepseek-v4-flash pp131072 (c1) 558.69 ± 0.00 558.69 ± 0.00 234608.36 ± 0.00 234603.87 ± 0.00 234621.63 ± 0.00
deepseek-v4-flash tg128 (c1) 19.10 ± 0.00 19.10 ± 0.00 23.00 ± 0.00 23.00 ± 0.00
deepseek-v4-flash pp131072 (c2) 548.87 ± 0.00 406.83 ± 132.39 360340.23 ± 117258.53 360335.75 ± 117258.53 360347.52 ± 117259.06
deepseek-v4-flash tg128 (c2) 1.05 ± 0.00 19.13 ± 0.22 25.00 ± 0.00 24.00 ± 1.00
deepseek-v4-flash pp131072 (c4) 546.73 ± 0.00 196.89 ± 56.72 602040.49 ± 121723.14 602036.01 ± 121723.14 602053.75 ± 121723.14
deepseek-v4-flash tg128 (c4) 0.70 ± 0.00 20.11 ± 1.47 25.00 ± 0.00 24.00 ± 1.22
deepseek-v4-flash pp131072 (c1) 573.71 ± 0.00 573.71 ± 0.00 228466.93 ± 0.00 228462.44 ± 0.00 228473.65 ± 0.00
deepseek-v4-flash tg512 (c1) 18.50 ± 0.00 18.50 ± 0.00 24.00 ± 0.00 24.00 ± 0.00
deepseek-v4-flash pp131072 (c2) 531.49 ± 0.00 409.53 ± 143.78 365049.44 ± 128158.79 365044.96 ± 128158.79 365059.40 ± 128161.25
deepseek-v4-flash tg512 (c2) 3.62 ± 0.00 18.88 ± 0.88 26.00 ± 0.00 25.00 ± 1.00
deepseek-v4-flash pp131072 (c4) 526.27 ± 0.00 188.42 ± 54.45 631612.72 ± 130990.99 631608.23 ± 130990.99 631626.03 ± 130991.41
deepseek-v4-flash tg512 (c4) 2.09 ± 0.00 19.28 ± 0.45 26.00 ± 0.00 25.00 ± 1.22
deepseek-v4-flash pp162816 (c1) 534.93 ± 0.00 534.93 ± 0.00 304375.99 ± 0.00 304371.51 ± 0.00 304384.97 ± 0.00
deepseek-v4-flash tg128 (c1) 20.62 ± 0.00 20.62 ± 0.00 24.00 ± 0.00 24.00 ± 0.00
deepseek-v4-flash pp162816 (c2) 521.46 ± 0.00 387.00 ± 126.26 470838.82 ± 153616.52 470834.33 ± 153616.52 470847.89 ± 153616.37
deepseek-v4-flash tg128 (c2) 0.81 ± 0.00 19.09 ± 0.42 24.00 ± 0.00 24.00 ± 0.00
deepseek-v4-flash pp162816 (c4) 519.15 ± 0.00 186.62 ± 53.53 789169.74 ± 158960.31 789165.25 ± 158960.31 789174.99 ± 158955.06
deepseek-v4-flash tg128 (c4) 0.54 ± 0.00 19.86 ± 0.79 25.00 ± 0.00 24.00 ± 1.22
deepseek-v4-flash pp162816 (c1) 542.47 ± 0.00 542.47 ± 0.00 300144.05 ± 0.00 300139.56 ± 0.00 300160.34 ± 0.00
deepseek-v4-flash tg512 (c1) 18.50 ± 0.00 18.50 ± 0.00 24.00 ± 0.00 24.00 ± 0.00
deepseek-v4-flash pp162816 (c2) 508.47 ± 0.00 388.37 ± 134.13 476007.57 ± 164392.18 476003.08 ± 164392.18 476017.56 ± 164391.67
deepseek-v4-flash tg512 (c2) 2.87 ± 0.00 17.99 ± 0.36 24.00 ± 0.00 23.00 ± 1.00
deepseek-v4-flash pp162816 (c4) 495.46 ± 0.00 207.66 ± 42.84 818907.10 ± 168931.83 818902.61 ± 168931.83 818912.38 ± 168926.54
deepseek-v4-flash tg512 (c4) 1.98 ± 0.00 18.75 ± 0.49 28.00 ± 0.00 25.25 ± 1.64

What stood out to me is that this thing stays surprisingly consistent at long context on a single Spark. The prefill and tg numbers don’t collapse the way you might expect as you stretch from 4K to 162K, and that was the whole point of the test.

Next up I’ll post the 180B REAP benchmarks too, and if the hardware cooperates I want to try longer contexts, maybe up to 500K.

submitted by /u/Dry-Tough-8068
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA