Improved TPS of Gemma 4 31B : the journey and also creating custom patches with VLLM fork

I improved the TPS of Gemma 4 31B. Improving TPS and performing optimisations requires understanding of the model architecture, and I had to fork VLLM and apply my own patch to break into making a configuration work as per my idea. I wrote a full article so that even beginners can understand and improve TPS for any model, this will serve as a mental model for anyone who start with TPS optimisations.

Also I strongly believe being just a mecha pilot wont be enough for inference optimisations. I had to go deep dive into the codebase and architecture for the solution to work.

P.S: Wrote in my own words.

https://medium.com/@abhijithneilabraham/learning-inference-how-to-host-and-improve-the-token-speed-of-an-llm-cff5623ab505?postPublishedType=initial