Humaneval benchmark for Deepseek V4 Flash 0731 vs GLM5.3 Flash on 2x DGX Spark setup

I have a 2 DGX Spark setup recently and I have been happily running Deepseek V4 Flash 0731. Since the release of GLM5.3 Flash and Qwen 3.8 Flash Next this week, a lot of folks are still waiting to see what model to run given their own hardware situations. I am very interested in running a GLM model locally and looks like nvfp4 would be a good option for my setup, but I have been hearing a lot of conflicting opinions (mostly negative) about GLM5.3 Flash on nvfp4 quant which gave me pause. So I decided to do a simple benchmark myself and hopefully this is useful for folks with the same setup:

Deepseek V4 Flash 0731 recipe : official checkpoint, fp8 kv, 1M context, 4 concurrent streams

GLM 5.3 Flash recipe : NVFP4 quant & Dflash2 drafter, fp8_e4m3 kv, 256k context, 6C

So raw numbers tell you GLM 5.3f is a decent upgrade over DSv4f 0731 especially with thinking enabled. Unsloth saying their Q4 quant has around 92% accuracy but looks like nvfp4 still holds up pretty well (97% would have been a SOTA score not that long ago and this is not even max thinking). The major trade off is the 256 context. I am pretty sure 512GB+ VRAM (or 4x sparks) people will be able to run the fp8 model + 1M context without issues and I am jealous 🥹 Regardless, your own experience matters more than any benchmark out there.