llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 21d ago
b10329
server, ui: only offer a working directory when a tool reads it ( #26762 ) The working directory chip showed up as soon as the server exposed any builtin tool, so a server started with just get_datetime, or a user who turned every filesystem tool off in the settings, still got a…
8 -
b10328
server: add initial tool isolation support (via docker) ( #26507 ) server: add initial tool isolation support (via docker) add docs adapt get_info py: fix type check cont separate tools_io_sandbox / tools_io_docker rename sandbox --> isolate x-tool-docker --> x-tool-runtime…
15 -
b10327
CUDA: fix thread/block count in quantized cpy kernel launches ( #26731 ) CUDA: fix thread/block count in quantized cpy kernel launches tests: add uneven block count cpy case Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
31 -
b10326
tts: account for the vocoder pass in the timings line ( #26733 ) get_output runs the waveform work the pipeline defers to it, from a single trailing window to a full pass depending on the model. Measuring it keeps the reported total and the audio to process ratio honest.…
6 -
b10322
sycl: coalesce the ssm_conv window loads ( #26612 ) test-backend-ops perf -o SSM_CONV on an Arc Pro B70, interleaved A/B against master, 6 reps, us/run: ne_a=[515,3328,1,1] ne_b=[4,3328,1,1] n_t=512 97.68 -> 52.95 1.85x ne_a=[937,8192,1,1] ne_b=[4,8192,1,1] n_t=934 516.16 ->…
33 -
b10321
metal : fix NORM/RMS_NORM for row lengths that leave a partial simdgroup ( #26708 ) ggml_metal_op_norm sized the threadgroup with nth = std::min(nth, args.ne00_t) , which can leave nth not a multiple of the simdgroup size. The kernels finish their row reduction with a…
7 -
b10319
mtmd: fix longest_edge ignoring min/max pixels ( #26638 ) mtmd: fix longest_edge ignoring min/max pixels nits Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
32 -
b10318
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
13 -
b10313
server: (router) add LRU scheduler ( #26572 ) add lru_sched handle coalescing (req leaves waiting queue) add tests fix stream case address review comments Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
35 -
b10312
server: (router) do not evict busy models ( #26567 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
37 -
b10311
mtmd: stop feeding the text stream again during Qwen3-TTS generation ( #26706 ) The reference implementation has two mutually exclusive prompt layouts. In non streaming mode the prefill carries the whole utterance text plus tts_eos summed with codec_pad, and the trailing text…
29 -
b10310
ggml : add aarch64 HWCAP fallbacks and fix fp16 variant detection ( #25554 ) ggml : add fallback definitions for missing aarch64 HWCAP bits ggml : require HWCAP_ASIMDHP for the aarch64 fp16 cpu variants Also rename has_fp16_va to has_fp16, the field gates the whole FEAT_FP16…
18 -
b10308
Mitigate crashing issue on Windows MSYS2 UCRT64 environment (GCC 16.1.0) ( #26555 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
16 -
b10307
sycl: fix UE4M3 parsing ( #25608 ) The NVFP4 quantization format stores a scaling factor for every group of 16 weights, packed into a single UE4M3 byte. The SYCL GPU code was converting these scale values using the E4M3 path, but that's signed , and these are unsigned values.…
13 -
b10306
sycl: *glu flat path ( #26354 ) tests: add SWIGLU perf cases perf mode had no GLU coverage. Adds SWIGLU at 17408 columns, 512 and 2048 tokens, f16 and f32, with the operands both fused and split. sycl: consolidate fused-GLU kernels They differed only in which op_* they called,…
14 -
b10305
sycl : Support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PRE ( #26568 ) support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PREwq update ops.md fix format issue Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
8 -
b10303
sycl : fix error Error OP FLASH_ATTN_EXT on arc770 ( #26441 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
17 -
b10301
cuda: fix warnings for unused variable/function ( #26688 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
36 -
b10299
metal : avoid threadgroup matrix array instantiation in kernel_lightning_indexer ( #26646 ) In MSL, declaring an array of matrix types like threadgroup half4x4 causes a 'no matching constructor' compilation error because MSL matrix types do not have zero-argument default…
17 -
b10298
mtmd: add chunk save/load function ( #26645 ) mtmd: add chunk save/load function nits add tests rn _MAX --> _COUNT Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
11 -
b10297
server: fix empty response for /cors-proxy ( #26656 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
10 -
b10295
model-loader : fix quantized reshaped tensor strides ( #26672 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
32 -
b10293
ci : onboard AMD ROCm CI with gfx1151 fixes ( #26544 ) ci: prepare for amd rocm ci Signed-off-by: Aaron Teo aaron.teo1@ibm.com ci: fix editorconfig-checker Signed-off-by: Aaron Teo aaron.teo1@ibm.com ci: fix device not recognised Signed-off-by: Aaron Teo aaron.teo1@ibm.com ci:…
23 -
b10291
vulkan: fix submission batching size, add debug tools for diagnosing causes of DeviceLost drivers errors ( #26371 ) vulkan: add debug tooling to get more information about a DeviceLost error fix submission threshold applied too late use logging macros, throw instead of aborting…
18 -
b10290
mtmd/ggml: add ggml_build_forward_order ( #26649 ) ggml: add ggml_build_forward_order ggml_build_forward_expand marks the tensor and all its ancestors for compute, so using it as a pure ordering hint (keeping q, k and v together) defeats ggml_build_forward_select: the unselected…
32 -
b10289
server: harden the file_glob_search directory walk ( #26626 ) server: don't walk Windows junctions in file_glob_search std::filesystem reports a junction as a plain directory, so the symlink guard misses it and a junction pointing back at an ancestor is walked until the path…
13 -
b10288
tests: re-enable MiniMax M3 in test-llama-archs ( #26633 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
14 -
b10287
mtmd: Unlimited-OCR fix max_tiles, setting in converter ( #25614 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
23 -
b10286
grammar : degrade max repetition >= 2000 to unbounded ( #26613 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
5 -
b10285
mtmd: support multi-row batching for deepseek-ocr ( #26154 ) mtmd: support multi-row batching for deepseek-ocr mtmd: weave deepseek-ocr rows in one shot instead of per row ( #26615 ) Co-authored-by: Saba Fallah sabafallah@gmail.com Website: https://llama.app macOS/iOS: macOS…
15 -
b10284
fit: Fix memory allocation for MTP layers ( #26605 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
29 -
b10282
server: Adding spec-decode counters to /metrics endpoint ( #26389 ) server: add spec-decode counters to /metrics endpoint server: fixed review comments and now aligned param names exactly with vLLM. Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
28 -
b10280
vendor : apply patches for subprocess.h ( #26606 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
29 -
b10278
build : remove GGML_METAL_USE_BF16 from all build scripts ( #26604 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
30 -
b10276
Prefer npm ci over install for security ( #26601 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
13 -
b10275
server: decode Windows OEM output to UTF-8 in built-in tools ( #26597 ) a child process writes in the OEM code page, which is not UTF-8 on a western Windows install, so accented output reaches the JSON layer as invalid bytes and gets replaced there, silently losing the…
38 -
b10274
mtmd: correcting duplicate empty audio chunks for short inputs ( #26536 ) correcting duplicate empty audio chunks for short inputs tests.sh code restored Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
30 -
b10273
sampler : remove "full-context windows" from history-based samplers ( #26524 ) Resolve -1 to 1024 instead of ctx-len for samplers Because of backend-sampling we initialize samplers before the complete llama_context is there. Therefore, we cannot infer the resolved context length…
36 -
b10271
ui: CWD for agent ( #26518 ) server : extend file_glob_search for UI pickers ui : add per-conversation working directory with picker ui : add path navigation and search scope to cwd picker Treat path-like queries (starting with / or ~) as directory navigation instead of…
15 -
b10270
mtmd: support Qwen3-TTS (note: breaking change to llama-tts binary) ( #26254 ) convert text model main model load ok convert encoder ok speaker encoder loading ok speaker enc graph adapt vocab for backbone (with some tricks) add suppress_tokens poc new mtmd gen api convert…
17 -
b10269
models : fix dflash wo_a reshape on load ( #26577 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
34 -
b10268
ci: fix pre-built binaries no longer working on macOS 15 and below ( #26375 ) ci: fix pre-built binaries no longer working on macOS 15 and below ci: add macOS deployment target to disabled KleidiAI build Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS…
5 -
b10267
speculative : refactor enabled configs common_speculative_init ( #26510 ) This commit contains a suggestion to reduce some code duplication in common_speculative_init when adding the enabled speculative decoding configurations. No tests were added but the existing server tests…
8 -
b10265
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
22 -
b10262
vulkan backend ops: implemented GATED_LINEAR_ATTN ( #25601 ) vulkan : add GATED_LINEAR_ATTN op docs : update Vulkan ops vulkan : remove unused GLA spec constant Updated ops.md ops.md update Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
29 -
b10261
vocab : validate plamo2 byte tokens ( #26511 ) validate plamo2 byte tokens --typo Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
32 -
b10259
model : allow reshape of tensors during load ( #26531 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
9 -
b10258
llama : move n_vocab from llama_sampler_data to penalty_sampler ( #26520 ) This matches how it is done for logit_bias and mirostat samplers, see #25262 (comment) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
13 -
b10256
sycl: parallelize the non-contiguous concat kernel ( #25852 ) sycl: parallelize the non-contiguous concat kernel Launch geometry only: the non-contiguous concat kernel launched a single-lane work-group (1, 1, 1), now it will launch a (1, 1, SYCL_CONCAT_BLOCK_SIZE) one.…
30 -
b10255
Extended SYCL oneDNN SDPA to non-FP16 KV caches (Q4_0–Q8_0 and FP32) ( #25874 ) sycl: extend oneDNN SDPA to Q4_0-Q8_0 and F32 KV caches Extends the oneDNN SDPA path (PR #25222 ) to handle non-F16 KV caches by dequantizing or converting K/V to dense FP16 on-device before feeding…
4