llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 4d ago
b10629
ci : update OS used for ROCM to Ubuntu 24.04 ( #27681 ) This matches what other build targets use and also what AMD advertises wheels as supporting. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42957551 macOS/iOS: macOS Apple…
30 -
b10628
rpc: support apple RDMA as an RPC transport ( #26421 ) rpc: support apple RDMA as an RPC transport remove set_tensor micro optimization, rpc socket pinning per CR remove transparent reconnect trigger apple builds on RPC changes Co-authored-by: Ryan Churaman rschu@meta.com…
21 -
b10625
chat : scope qwen3-coder workarounds ( #27679 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42919681 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
21 -
b10622
metal : null-check buffer alloc to fix OOM crash ( #25371 ) metal : null-check ggml_metal_buffer_init result to avoid OOM crash ggml_backend_metal_buffer_type_alloc_buffer used the result of ggml_metal_buffer_init without checking for NULL. ggml_metal_buffer_init returns NULL…
27 -
v0.3.0
Overview llama.cpp 0.3.0 introduces the dots3-note multimodal model (with a new DSA-ISWA KV cache), MTP support for GLM-4.5-Air, and tensor-split ( -sm tensor ) plus multi-sequence rollback fixes for DeepSeek 4. ggml is bumped to v0.22.0 (meta-backend tensor split, per-op Metal…
23 -
b10621
llama.cpp : bump version to 0.3.0 ( #27696 ) llama.cpp : bump version to 0.3.0 ci : update release default desc scripts : add prompt for generating release summary Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42818481 macOS/iOS:…
32 -
b10620
sync : ggml Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42810281 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
12 -
b10618
grammar : parse - in char classes as literal hyphen ( #27591 ) grammar : accept "-" escape in character classes gbnf_escape_char_class() escapes '-' as "-" but parse_char() rejected that escape, so generated tool-call grammars failed to parse. Assisted-by: Claude Code…
21 -
b10617
sycl : mark tq2_0 as not supported ( #27660 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42777546 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
24 -
b10615
metal : per-device tuned (Q, NE) for flash-attn vec ( #26570 ) metal : per-device tuned (Q, NE) for flash-attn vec ( #25750 ) rebase Q-generic FA vec body from 01dc936 ( #23114 ) add 53 f16 (Q,NE) flash-attn vec instantiations (vec 80 -> 133) add FA vec (Q,NE) tuning table +…
28 -
b10614
metal: per-op source split + parallel compile ( #26561 ) metal : per-op source split + parallel compile ( #24021 ) preliminary extract common header op source split split metallib into 8 libs && load in parallel derive kernel->library routing from functionNames x-macro lib list…
21 -
b10612
tests : disable DOTS3NOTE arch test for WebGPU ( #27654 ) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42591112 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
26 -
b10610
ggml : shorten virtual device naming in CUDA and Metal ( #27608 ) ggml : shorten virtual device naming in CUDA and Metal Assisted-by: llama.cpp:DeepSeek-V4-Flash-0731 ggml-metal : build device description at init Assisted-by: llama.cpp:DeepSeek-V4-Flash-0731 cont : naming…
22 -
b10606
ggml : fix ggml_clamp ( #27644 ) ggml : fix ggml_clamp cont : update ggml-alloc Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42576162 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
28 -
b10604
Deepseek 4: -sm tensor ( #26490 ) DSV4: sm tensor set coarser granularity for head splits fix dspark add model saving for dsv4 + allow dflash to return on specific device add comment about dsv4 seq_rm simplify add shared expert delayed allreduce remove special test for dsv4…
35 -
b10603
model : support MTP in GLM-4.5-Air ( #26534 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42447665 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
25 -
b10599
test: move tools/parser to tests ( #27548 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42442638 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
4 -
b10598
mtmd: use pillow-accurate algo, correct resize_algo for all models ( #27594 ) mtmd: use pillow-accurate resize algo, correct resize_algo for all models speed optimization Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42440726…
20 -
b10595
server : add LLAMA_SERVER_SLOTS_N_DIFF ( #27600 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42423433 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
23 -
b10594
common : skip device_info loop if it's not going to be printed ( #26692 ) The device_info loop iterates over the discovered devices and gets the available and total memory counts. With the CUDA backend (and possibly others too) this requires creating a GPU context, which, in…
11 -
b10593
DeepseekV4: fix rollback with multi-seq ( #26756 ) DeepseekV4: fix rollback with multi-seq fix model loading make pending rollback single use only clear cache for seq_id for full load add assert for compress ratio make graph topology static pass true instead of flags in…
25 -
b10590
vendor : update subprocess.h ( #27409 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42402532 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
17 -
b10589
cuda : add POOL_1D support ( #27573 ) cuda : add POOL_1D support fix: add missing trailing newline for editorconfig compliance Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42401257 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
29 -
b10588
common: json.h: fix clang lto ( #27575 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42368523 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
11 -
b10587
vulkan : added the PAD_REFLECT_1D operation ( #26586 ) vulkan : added PAD_REFLECT_1D operation Implemented the GGML_OP_PAD_REFLECT_1D operation for the Vulkan backend Changes: pad_reflect_1d.comp: implemented the GLSL compute shader with reflection logic vulkan-shaders-gen.cpp:…
9 -
b10586
mtmd: use ggml_rope_set_offset ( #27521 ) mtmd: use ggml_rope_set_offset add comment Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42334609 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
24 -
b10585
common: add json.h abstraction ( #27511 ) add common/json migrate common adapt jinja migrate server big wip migrate tests wip revert some excessive changes wip wip 2 revert redundant changes fix server crash various fixes fix ci harden a bit clean up rm json-shim add some…
26 -
b10584
fit: also take into account n_streams ( #27496 ) fit: also take into account n_streams server: make the draft context follow the target context With a non-unified KV cache the target context now holds n_ctx_train tokens per sequence, while the draft context was still created…
12 -
b10582
ci : Restore ROCm job for Ubuntu ( #27399 ) Revert "ci : disable ubuntu-rocm ( #26969 )" This reverts commit 9558fa4 . ci: set ccache compiler_check=content for ROCm build The ROCm toolchain is pip-installed fresh on every run, so the clang binary's mtime changes each time. With…
36 -
b10581
model : support DSpark for bailingmoe3 ( #27508 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42309942 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
17 -
b10580
mtmd: support dots3-note vision+audio ( #27524 ) text: conversion init impl mtmd: conversion impl mtmd cpp Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co…
22 -
b10578
ggml: optimize concat op by replacing per-element memcpy with row-level memcpy ( #24575 ) ggml: optimize concat op by replacing per-element memcpy with row-level memcpy ggml: fix concat offsets for row-level copies ggml: add concat row contiguity asserts ggml: move concat block…
24 -
b10569
model: add dots3-note ( #27060 ) text: conversion init impl address review comments fix rope move to a new llama_kv_cache_dsa_iswa Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42267121 macOS/iOS: macOS Apple Silicon (arm64) macOS…
9 -
b10568
model: use ggml_rope_set_offset() ( #27382 ) model: use ggml_rope_set_offset() partially apply to deepseek2 Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42252109 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
8 -
b10567
ci : run ccache-clear as the last step of release jobs ( #27503 ) ci : run ccache-clear as the last step of release jobs Assisted-by: pi:llama.cpp/Qwen3.8-27B update disabled job too to force rebase Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Website:…
11 -
v0.2.0
Overview New version has been released. Nightly build: b10566 Web UI: the nightly-tag.txt asset contains the tag of the corresponding nightly release More info: dist : releases and versioning of ggml-org projects Changelog since v0.1.2 bb4caa7 llama.cpp : bump version to 0.2.0 (…
36 -
b10566
llama.cpp : bump version to 0.2.0 ( #27498 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42207505 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
22 -
b10549
TP: enable tensor split for LFM2/LFM2MOE ( #26993 ) Assisted-by: deepseek-v4-flash Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42096995 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
32 -
b10545
metal : clamp K extent in tensor API mat-mat kernel for K not a multiple of 32 ( #27450 ) The Tensor API mat-mat path of kernel_mul_mm (GGML_METAL_HAS_TENSOR) fed a static K=32 tile to the matmul2d op on every iteration. On the last, partial K tile (ne00 % 32 != 0) the src1…
25 -
b10541
mtmd: add --mmproj-device argument ( #23255 ) feat: add --mmproj-device arg & backwards compatible MTMD_BACKEND_DEVICE env var feat: load mmproj device backend immediately, add -mmdev shortflag fix: its a pointer now get the name clean up gen docs nits Co-authored-by: Xuan Son…
28 -
b10539
vulkan: FA MMQ should use fp32 for Q quantization calculations ( #27413 ) Codex found that qd could be a denorm and 1/qd would overflow. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42034841 macOS/iOS: macOS Apple Silicon (arm64)…
30 -
b10538
metal : dequant kv cache only for large batches ( #27438 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/42031082 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
5 -
b10537
CI: Use LLVM's OpenMP over MSVC_DEBUG_non_redist on Windows ( #26678 ) CI: Use LLVM's OpenMP over MSFT_DEBUG_non_redist on Windows Currently, we ship the non-redist debug version of microsoft's libomp. This PR changes this to official LLVM's release, also packaging the license…
35