50% tg increase with offloading "hot" experts to VRAM

I got a 50% performance boost (20 t/s -> 30 t/s) in llama.cpp for MoE models that don’t fit entirely in VRAM—in my case, Qwen 3.8 Flash Next.

The idea is simple: instead of offloading entire layers to the GPU, I offload only the “hot” experts. I found that certain groups of experts remain relatively stable across coding, refactoring, and code-review workloads.

https://github.com/timadinorth/llama.cpp/pull/1

A couple of important caveats: this llama.cpp fork has been tested only on coding workloads, and it’s useful only when the full model cannot fit in VRAM.
Will upstream ever accept it? Probably not. Opus did the low-level implementation, and I don’t feel like showing up to explain every line