Sharp template to NInfer: -42% output tokens, same speed
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
Sharp is u/peculiar-ragdoll's system prompt that makes Qwen answer way more tersely without losing correctness. It's built on top of froggeric's fixed chat templates for Qwen; several fixes now in the v22.x templates (error-escalation tiers, false retry-loop kills, multi-system merging, correct tool-arg serialization) originated in u/Chromix_'s improved Jinja template before being merged upstream.
NInfer is a hyper-tailored inference engine that only runs certain Qwen models on 5090. NInfer doesn't support changing Jinja templates, so I overlaid the behavior in C++ instead in a fork: ninfer-sharp
--chat-style sharp-v22.1appends Sharp's terseness instruction to the system prompt--reasoning-effortwith 7 levels,none= thinking off- Official model artifact untouched (NInfer verifies templates by hash)
- Pinned to Sharp v22.1 semantics — the overlay replicates the terse block and effort mapping, not the later v22.2/v22.3 tool-path changes (tool-call format stays NInfer's own)
In my tests, the output remained byte-identical to the actual Sharp Jinja template except the engine's internal thinking markers, which it depends on for parsing. Numbers (5090, Qwen3.8 27B, xhigh (default)):
| default | sharp |
|---|---|
| completion tokens | baseline |
| wall time | baseline |
| decode speed | baseline |
[link] [comments]
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.