r/LocalLLaMA · · 1 min read

Sharp template to NInfer: -42% output tokens, same speed

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

Sharp is u/peculiar-ragdoll's system prompt that makes Qwen answer way more tersely without losing correctness. It's built on top of froggeric's fixed chat templates for Qwen; several fixes now in the v22.x templates (error-escalation tiers, false retry-loop kills, multi-system merging, correct tool-arg serialization) originated in u/Chromix_'s improved Jinja template before being merged upstream.

NInfer is a hyper-tailored inference engine that only runs certain Qwen models on 5090. NInfer doesn't support changing Jinja templates, so I overlaid the behavior in C++ instead in a fork: ninfer-sharp

  • --chat-style sharp-v22.1 appends Sharp's terseness instruction to the system prompt
  • --reasoning-effort with 7 levels, none = thinking off
  • Official model artifact untouched (NInfer verifies templates by hash)
  • Pinned to Sharp v22.1 semantics — the overlay replicates the terse block and effort mapping, not the later v22.2/v22.3 tool-path changes (tool-call format stays NInfer's own)

In my tests, the output remained byte-identical to the actual Sharp Jinja template except the engine's internal thinking markers, which it depends on for parsing. Numbers (5090, Qwen3.8 27B, xhigh (default)):

default sharp
completion tokens baseline
wall time baseline
decode speed baseline
submitted by /u/xrailgun
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA