Following up on my previous post about my budget server setup (Intel N100 + RTX 5060 Ti 16GB), a few of you asked for a deeper dive into my actual inference config and real-world agentic performance.

Like many of you, I was refreshing the page waiting to download Qwen 3.8 27B the second it dropped. After spending the entire weekend stress-testing it with agentic coding workflows, I managed to run a complete, large-scale project almost entirely autonomously (over 1M total tokens processed, only 3 prompts total).

Here is a quick breakdown of the core setup before we dive into the config and workflow details.

Quick Specs & Params

• Model: Qwen3.8-27B-UD-Q3_K_XL.gguf
• Hardware: RTX 5060 Ti (16GB VRAM) + Intel N100 (4C/4T, 16GB RAM)
• Context Window: 73,728 (73k context) running comfortably in 16GB VRAM!
• KV Cache Quant: q4_1 for main context, q5_1 for MTP draft context
• Speculative Decoding: Native MTP enabled (spec-type = draft-mtp, n-max = 2)
• Sampling: temp = 0.4, top_p = 0.90, top_k = 15, min_p = 0.02

---

The Experiment: Building a full API in 3 Prompts

Instead of running synthetic benchmarks, I put this setup through a real-world software engineering pipeline: building an unofficial REST API and MCP Server for a legacy vBulletin forum.

1. Prompt 1 (Site Architecture & Analysis): Asked the model to map out the target site. It generated a flawless ~1,500-lines Markdown spec covering structural analysis, scrapable HTML nodes, expected JSON payloads, stack selection, pagination logic, session auth, and search endpoints—far more thorough than I would have written manually.
2. Prompt 2 (Development Architecture): Using the spec as the single source of truth, it designed a modular NestJS API implementation plan broken into 9 execution phases:
3. Phase 1: Project Scaffolding
4. Phase 2: Domain Models
5. Phase 3: Scraping Core (HTTP + Rate Limiting + Retries)
6. Phase 4: HTML Parsers (cheerio)
7. Phase 5: Cache Layer
8. Phase 6: Application Services + REST API
9. Phase 7: Authentication (Cookie Sessions)
10. Phase 8: MCP Server (Primary Deliverable)

11. Phase 9: Hardening, Docs, & Delivery

12. Prompt 3 (Autonomous Agentic Execution): The real test. I instructed OpenCode (using Qwen 3.8 27B) to act strictly as an orchestrator, spawning sub-agents for each task phase. It ran autonomously for ~2 hours. When context limits were approached, OpenCode summarized its state and kept building. It wrote unit tests, enforced linting, and delivered fully functional code—only needing one minor automated fix when fed a edge-case raw HTML payload.

---

The llama.cpp Configuration File

Here is my exact --models-preset router configuration file. Note how fit = off is used on the 27B profile alongside ctx-size = 73728 (73k) and q4_1 KV cache quantization to maximize VRAM allocation while preserving native MTP performance.

```ini

==============================================================================

LLAMA.CPP — INFERENCE CONFIGURATION (router mode / --models-preset)

==============================================================================

Hardware Target:

GPU: 16 GB VRAM (RTX 5060 Ti)

CPU: Intel N100, 4C/4T (Debian Headless)

------------------------------------------------------------------------------

GLOBAL / BASELINE

------------------------------------------------------------------------------

[*]

--- CPU THREADING -----------------------------------------------------------

Reserve 1 core for OS/services during decode.

Use all 4 threads during prompt prefill bursts.

threads = 3 threads-batch = 4

--- SERVER / CONCURRENCY ---------------------------------------------------

Single slot, disabled continuous batching for maximum single-user throughput.

parallel = 1 cont-batching = 0

--- GPU / VRAM FIT ---------------------------------------------------------

flash-attn = on fit = on

Safety headroom for VRAM physical limit (MiB).

Set low (128) because system is headless (100% VRAM available for inference).

NOTE: If using MTP draft KV caches, watch out for double VRAM allocation.

Bump to 128-256 if you encounter OOMs.

fit-target = 128

--- CONTEXT & CACHING ------------------------------------------------------

ctx-size = 65536 context-shift = 1

Disable context checkpoints (avoids reprocessing issues in hybrid architectures)

ctx-checkpoints = 0

RAM Prompt Cache (2 GiB)

cache-ram = 2048

--- GLOBAL KV CACHE --------------------------------------------------------

cache-type-k = q5_1 cache-type-v = q5_1

--- PREFILL / BATCHING -----------------------------------------------------

batch-size = 2048 ubatch-size = 1024

--- DEFAULT SAMPLING (Coding / Precision) ----------------------------------

temp = 0.2 top-p = 0.95 top-k = 20 min-p = 0.0 repeat-penalty = 1.0 presence-penalty = 0.1 frequency-penalty = 0.0

------------------------------------------------------------------------------

QWEN 3.8 27B — REASONING & HEAVY CODING PROFILE

------------------------------------------------------------------------------

[qwen3.8-27b] model = /opt/llama-infrastructure/models/Qwen3.8-27B-UD-Q3_K_XL.gguf

fit = off ctx-size = 73728 context-shift = 1

Native Model MTP (Speculative Decoding)

spec-type = draft-mtp spec-draft-n-max = 2 spec-draft-p-min = 0.85

KV Quantization (q4_1 allows us to fit 73k context in 16GB VRAM)

cache-type-k = q4_1 cache-type-v = q4_1 cache-type-k-draft = q5_1 cache-type-v-draft = q5_1

Thinking / Reasoning Budget Params

chat-template-kwargs = {"preserve_thinking": true, "reasoning_effort":"medium"} reasoning-budget = 5000

Reduced batch sizes to prevent VRAM spikes during massive prefills

batch-size = 1024 ubatch-size = 512

Official / Recommended Quant Sampler Tuning

temp = 0.4 top-p = 0.90 top-k = 15 min-p = 0.02

```