llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 9d ago
b10534
CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover ( #26079 ) CUDA: runtime GGML_CUDA_MMVQ_MAX to tune the mvq->MMQ decode crossover Add a runtime override of the mul_mat_vec_q -> MMQ batch crossover (default MMVQ_MAX_BATCH_SIZE). Lowering it…
30 -
b10533
common : gracefully fallback on unsupported regex patterns in JSON schema ( #26939 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42018860 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
16 -
b10532
metal : dequantize quantized KV to F16 before flash attention ( #27390 ) metal: dequantize q8_0 KV to f16 before flash attention Add a preprocessing pass for GGML_OP_FLASH_ATTN_EXT on the Metal backend: when the KV cache is quantized (Q8_0 for now), dequantize K and V into a…
34 -
b10531
Revert "tensor-split meta backend fixes ( #26502 )" ( #27433 ) This reverts commit d59d455 . Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42012577 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
14 -
b10520
ggml-cpu: gate __fp16 on __ARM_FP16_FORMAT_IEEE ( #26860 ) ggml-cpu: gate __fp16 on __ARM_FP16_FORMAT_IEEE __ARM_NEON only signals NEON availability. The __fp16 type also needs the IEEE half format, implied on AArch64 but selected with -mfp16-format=ieee on 32 bit Arm, where the…
14 -
b10519
server: refactor sleep handling, allow access /metrics during sleep ( #27376 ) add cached responses refactor on_sleeping_state allow accessing metrics during sleep metrics task should not reset timer updated docs fix fix get_res_model_info add test fix a race condition split…
32 -
b10517
vulkan : dequant q8_0 KV once in coopmat1 ( #25494 ) vulkan : dequant q8_0 KV once in coopmat1 Assisted-by: Claude (Opus 4.8) vulkan : fall back instead of aborting when FA scratch exceeds maxStorageBufferRange vulkan : require KV-cache layout in FA dequant path Assisted-by:…
19 -
b10516
vulkan: add null checks in ggml_vk_queue_command_pools_cleanup ( #27353 ) Guard against null queue pointers. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/41873046 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
4 -
b10514
model : GraniteSWAForCausalLM / GraniteMoeSWAForCausalLM ( #25505 ) feat(convert): Add conversion for GraniteSWAForCausalLM Branch: GraniteSWAForCausalLM AI-usage: full (Bob, OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart ghart@us.ibm.com feat(llama): Add granite_swa…
38 -
b10509
ggml: add ggml_rope_set_offset (+ metal support) ( #27120 ) add params cpu kernel metal kernel add test backend ops gate other backends ggml: (cuda) support ggml_rope_set_offset ( #27121 ) rm cuda supports_op guard, fix webgpu clang-format ggml: support ggml_rope_set_offset on…
29 -
b10524
opencl: make the MoE expert scatter deterministic ( #26464 )
17 -
b10507
mtmd: add mtmd_bitmap_set_mergeable ( #27348 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
11 -
b10506
metal : dequantize q8_0 using packed types ( #27370 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
24 -
b10505
server: add dedup-cache-models preset option ( #27346 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
30 -
b10502
ci : add attestation for signed release artifacts ( #25933 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
21 -
b10499
server: (cosmetic) do not print cmd_child_to_router messages [no rele…
31 -
b10488
ci : Update OpenVINO to 2026.3, skip nemotron-h rollback test ( #27292 ) update to ov-2026.3, update device drivers ci: skip nemotron-h rollback test on OpenVINO The OpenVINO backend does not support SSM_SCAN, so the Nemotron-H recurrent state rollback graph is split and cannot…
31 -
b10486
mtmd: fix LFM2 image tiling threshold ( #27057 ) mtmd: fix LFM2 image tiling threshold refactor testing fix fix on windows Co-authored-by: Xuan Son Nguyen son@huggingface.co Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
12 -
v0.1.2
Note Semantic versioning is still work in progress. More info can be found in https://github.com/ggml-org/ggml/discussions/1579 Nightly build: b10485 Change log since v0.1.1 1511ce3 sync : ggml da786dc ggml : bump version to 0.20.2 (ggml/1589) 27e345b build : fix xcframework +…
26 -
b10485
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
30 -
b10483
build : fix xcframework + cmake clean-up ( #27304 ) xcframework : fix build mtmd : remove unused include path vendor : use vendor::hash alias target in cmake CMake reserves "::" in target names for imported/alias targets, so the real target keeps the name vendor-hash and a…
14 -
b10481: CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark (#26843)
CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark Signed-off-by: ynankani ynankani@nvidia.com skip moe experts and allow others based on k geometry (allow only small idle tail) Signed-off-by: ynankani ynankani@nvidia.com rename MMVQ DGX Spark params to GB10 and fix MSVC…
23 -
b10472
cuda : skip UMA override for HIP builds ( #27083 ) AMD APUs report accurate memory via hipMemGetInfo. Using MemAvailable over-promises on small-carveout systems. fixes #18159 Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
28 -
b10470
ci : push release tag explicitly in release.yml ( #27261 ) Add a "Create and push git tag" step to the release job, right before the "Create release" step. The tag is created with git tag and pushed with the deploy key already configured by the Clone step, instead of relying on…
21 -
b10456
sycl: fix thread/block count in quantized cpy kernel launches ( #27160 ) Adjusts the thread/block count to be proportional to the size of the quant, reducing under/over subscription. Largest perf improvement is the q4_0 -> f32 path, with, on a Arc 70, throughput goes from 20.21…
10 -
b10455
[SYCL] support OP OPT_STEP_ADAMW, OPT_STEP_SGD ( #25268 ) fix conflict fix conflict of ops.md fix conflict of ops.md update the ops.md Co-authored-by: Neo Zhang Jianyu jianyu.zhang@intel.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
7 -
b10453
model : remove some ggml_concat ( #27176 ) Co-authored-by: Xuan Son Nguyen son@huggingface.co Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu…
12 -
b10454: ci : fix dry-run reporting in make-release job [no ci] (#27167)
This commit fixes the reporting in the make-release CI job when --dry-run is used. It will currently incorrectly report that all checks pass even if there are steps that fail. Refs: #26839 (comment)
34 -
b10452
chat: refactor handling supports_string_content / supports_typed_content ( #27130 ) better supports_string_content cap detect test: add "skip" messages_inp_normalizer Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
10 -
b10451
llama : check LoRA tensor data is within file bounds ( #27056 ) llama : check LoRA tensor data is within file bounds Update src/llama-adapter.cpp Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co…
18 -
b10450
ui: mask API Key field in settings and error splash to stop browser a…
22 -
b10448
model: add Kimi-K3 text model ( #26185 ) model: add Kimi-K3 text model Hybrid KDA (linear) + MLA (full) attention as in Kimi-Linear-48B, plus five things that architecture does not have: cross-layer residual attention (attn_res_block_size) latent MoE (routed experts run at…
34 -
b10447
server: re-design yield_to_queue thread model ( #27133 ) run common_speculative_process in worker swap worker <--> main thread design Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
17 -
b10446
vendor: update BoringSSL to 0.20260813.0 ( #27099 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
11 -
b10444
common: support --models-dir loading MTP assistant models ( #24431 ) common: support --models-dir loading MTP assistant models common: preset: check for MTP models with strict prefix common: preset: Take advantage of PR #27005 handle other draft types drop eagle3 clean up…
33 -
b10443
fix: check gguf array type before reading ( #27075 ) fix: check gguf array type before reading update skill Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
22 -
b10442
vulkan: add SHMEM_STRIDE_PAD/APPLY_SLM_A_RESHAPE for coopmat1 on Intel Xe ( #25380 ) vulkan: add SHMEM_STRIDE_PAD/APPLY_SLM_A_RESHAPE for coopmat mul_mm on Intel Xe vulkan: fix shmem estimate for Intel SHMEM_STRIDE_PAD=0 in matmul_shmem_support cacheline aligned for shared…
8 -
b10441
common: migrate the deprecated --mmap/--no-mmap to --load-mode ( #26934 ) Replace the deprecated --mmap, --no-mmap, --mlock, and --direct-io flags with the unified --load-mode argument across scripts, examples, and documentation. Internal warning message and env var docs updated…
11 -
b10437
model : add support for MiniMaxText01ForCausalLM and MiniMaxM1ForCausalLM ( #27018 ) llama : support for MiniMax-Text-01 model chore : renames to match the other MiniMax models model : add logits mask as MiniMax-Text-01 embeddings tensor has zero-valued embeddings for tokens >=…
22 -
b10436
mtmd, common: various fixes ( #27071 ) apply fixes cont revert gguf fix Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu…
18 -
b10435
jinja : fix quadratic cost in gather_string_parts ( #27034 ) jinja : fix quadratic cost in gather_string_parts fix some comments remove test Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
6 -
b10438: mtmd: fix Granite4 Vision image sequence assembly (#26653)
mtmd: fix granite 4v grid assembly (cherry picked from commit 91f82eb ) mtmd: fix truncation for scaled image height and width before unpad Signed-off-by: Hemanth Battu hbattu@ibm.com mtmd: remove MTMD_DUMP_EMBD debug scaffolding Signed-off-by: Hemanth Battu hbattu@ibm.com clean…
23 -
b10434
chat : pass reasoning_effort to template chat: add reasoning_effort to common_chat_templates_inputs Store OpenAI Chat Completions reasoning_effort and make it available to jinja templates (with model specific translations where required). Assisted-by: llama.cpp:Muse-Glimmer-30B…
16 -
b10433
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
13 -
b10431
ggml : recurrent state rollback for ggml_ssm_scan ( #26623 ) Initial changes for Recurrent state rollback for nemotron for cpu and cuda Removing CPU RS rollback. Will enable it in subsequent PRs addition of test case Removing assert and calling runtime API to check if op is…
21 -
b10430
llama : allow virtual igpu devices ( #26953 ) llama : allow virtual igpu devices cont : better comment Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
26