llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 19d ago
b11182
llama : add llama_prec_policy + model-driven W4A4 path ( #24364 ) Rebase and update based on #26675 Signed-off-by: ynankani ynankani@nvidia.com CI failure fix(launh_bounds overload on HIP) and cleanup Signed-off-by: ynankani ynankani@nvidia.com Address review comments…
10 -
b11181
HIP: bump HIP_VERSION requried for fp8 to avoid missing __hip_fp8_e4m3 support in 6.2 ( #29231 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50216799 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
25 -
b11180
rpc: include nb in the get_alloc_size cache key and floor the result at ggml_nbytes ( #29283 ) rpc : include nb in the get_alloc_size cache key and floor the result at ggml_nbytes cont : remove redundant comment cont : add TODO Co-authored-by: Georgi Gerganov ggerganov@gmail.com…
17 -
b11179
[SYCL] support sparse FA ( #28796 ) fix conflict fix format issue rm unused code Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50188435 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
26 -
b11178
musa: fix PH1 (MTT S5000) operator failures and build issues ( #29193 ) musa: use 16-byte copies for MUSA like sm_70+ ggml_cuda_get_max_cpy_bytes() derives the copy width from CUDA_ARCH . mcc never defines it, so MUSA fell into the generic branch and returned 8 bytes instead of…
35 -
b11177
CUDA: fuse RMS_NORM + SCALE into one kernel ( #29393 ) #28068 builds the GDN q/k l2norm as ggml_scale(ggml_rms_norm(x, eps/n), 1/sqrt(n)). This adds 2 SCALE nodes per GDN layer, 96 extra kernel launches per ubatch on Qwen3.8-27B (48 GDN layers). The extra kernels take no…
38 -
b11176
llama : fix tensor split for fused qkv with uneven K/V head sizes ( #2 …
6 -
b11175
hexagon: add q5_k quant type support ( #29123 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50064037 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
8 -
b11173
metal : fix graph capture and handle empty graphs ( #29390 ) return early when the graph has no nodes drop the redundant reset of capture_compute: the decrement at the top of the function already transitions the counter from 0 to -1, so a capture happens exactly once hint at…
27 -
b11172
metal : optimize sparse FA + clean-up ( #29377 ) metal : cache sparse FA indices in shared memory Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp metal : simplify shared memory size calculation Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp pi : update general…
7 -
b11171
sync : ggml ( #29396 ) ggml : bump version to 0.25.2 (ggml/1642) ggml : fix ubsan error in ggml_graph_nbytes (ggml/1644) ggml : bump version to 0.25.3 (ggml/1645) sync : ggml Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50020953…
21 -
b11170
hexagon: handle multi-sequence in concat_2d ( #29344 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50014220 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
36 -
b11169
llama-grammar: fix numeric truncation for token_id parsing ( #29382 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50007794 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
21 -
b11168
hexagon: dynamic quantizer improvements ( #29395 ) hexagon: fix accuracy issue in Q8_0 N=1 MUL_MAT hex-quant: fix register spills hex-mm: use dma for all dyn.quant paths Co-authored-by: Aparna M P aparmp@qti.qualcomm.com hex-mm: remove obsolete run_quant_task hex-mm: update…
35 -
b11167
hexagon: support I32 CPY and CONT ( #29379 ) Assisted-by: OpenCode Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49994729 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
24 -
b11166
cuda : add F16 kernel support for CONV_2D_DW ( #29064 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49985945 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
12 -
b11165
test: flush status ( #28352 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49978694 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
12 -
b11163
llama: add llama_batch_ext ( #24669 ) (wip) add llama_batch_ext wip updated design updated impl change signature unused var demo common_prompt_batch_decode fix pos tmp disable test-batch-alloc fix compat nits: add const no more pos_max add comment about…
18 -
b11160
vulkan: int8 coopmat1 matmul implementation for AMD RDNA3 and RDNA4 ( #27952 ) vulkan: add int8 coopmat quantized matmul shader apply scales inline use scalar sums probe and directly access coopmat values instead of going through shmem add q8_0 support add BK_STEP to shader,…
11 -
b11159
vulkan: handle misalignment in conv_2d and conv_3d ( #29365 ) vulkan: handle misalignment in conv_2d and conv_3d fix test-backend-ops print Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49846182 macOS/iOS: macOS Apple Silicon (arm64)…
6 -
b11158
vulkan: tune KHR cooperative matrix support for Adreno GPUs ( #29328 ) Enable coopmat support for Vulkan backend Fixed the mul_mat_s Removed the debug statement Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49831613 macOS/iOS: macOS…
35 -
b11157
cuda : add conv3d with implicit GEMM ( #29137 ) cuda : add conv3d with implicit GEMM cuda : refine conv3d implicit GEMM and handle empty kernels Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49789544 macOS/iOS: macOS Apple Silicon…
21 -
b11156
model : add Ling 3.0 VL support ( #29151 ) model : fold Ling 3.0 VL into the BailingMoeV3 architecture Assisted-by: Scout model : keep shared NORM rope list intact when gating bailingmoe3 on mrope sections Co-authored-by: aetherbird aetherbird@users.noreply.github.com Website:…
36 -
b11155
server,common : fix the GCC 12 stringop-overread false positive (again) ( #29325 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49775037 macOS/iOS: macOS Apple Silicon (arm64) macOS…
34 -
b11154
test-save-load-state : print a per-model results table in --models mode ( #29316 ) test-save-load-state : print a per-model results table in --models mode in --models mode the output was very heavy: every model printed its token dumps, per-test headers and PASS lines. instead,…
35 -
b11153
hexagon: reject MUL_MAT_ID when src1 precision is F32 ( #29348 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49757881 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
11 -
b11151: convert : allow vision target for DFlash/Dspark (#29339)
Resolve the target arch with get_model_architecture so vision targets (e.g. Lfm2VlForConditionalGeneration) map to their text model for the vocab. Fix double rope reorder for LFM2/LFM2.5 DSpark drafters
17 -
b11149
tests: add -b/--backend option to test-llama-archs for testing a specific backend ( #27372 ) tests: add backend option to test-llama-archs Update tests/test-llama-archs.cpp Co-authored-by: Johannes Gäßler johannesg@5d6.de remove extra space Co-authored-by: Johannes Gäßler…
24 -
v0.5.0
Overview This release focuses on backend performance and correctness, broader model coverage, and more robust server/router operation. It adds HRM-Text (DFM Mimir 1B) support, MiMo-V2.6 and HunyuanOCR conversion support, ggml 0.25.0 backend improvements, multi-address HTTP…
26 -
b11147
opencl: add A8 Q6_K non-MoE dp4a binary kernel ( #29057 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49630180 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
26 -
b11146
llama.cpp : bump version to 0.5.0 ( #29333 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49623059 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
33 -
b11140
CUDA: enable sparse-fa for dsv4 prefill (again) ( #29298 ) CUDA: enable sparse-fa for dsv4 prefill (again) CUDA: unroll the query loop of the sparse mask scan The query loop of flash_attn_mask_to_sparse_indices has a runtime trip count, which keeps the unrolled scan over the…
35 -
b11139
server: fix token counting API crash on sleep ( #29309 ) server: wake up sleeping server correctly server: wake up sleeping server correctly (local aliases removed) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49582694 macOS/iOS:…
26 -
b11138
jinja : parse unary +/- before variables ( #29244 ) jinja : parse unary +/- before variables Lexer already emits unary_operator for -n / +n, and runtime executes unary -. Parse them at multiplicative precedence so slices like items[:-n] and GigaChat indent[:-indent_factor] work.…
8 -
b11136
server: accept OpenAI video_url content type and data: video URIs ( #27921 ) The OpenAI chat completions API specifies content part type "video_url" with a {"url": ...} object, and clients typically send data: URIs (e.g. data:video/mp4;base64,...). The llama-server only accepted…
11 -
b11132
model : support Gemma4 DSpark draft backbone ( #29226 ) dspark: add Gemma 4 draft support Add GGUF conversion and runtime support for full-attention and SWA Gemma 4 DSpark drafts, including tied output weights and boolean backbone metadata. Assisted-by: Codex dflash: infer Gemma…
8 -
b11130
make-release : update summary prompt Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49525233 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
30 -
b11126
vulkan: add IQ4_XS MMQ/MMV matmul kernels ( #28415 ) vulkan: optimize IQ4_XS matmul kernels Assisted-by: OpenAI Codex vulkan: address IQ4_XS review nits drop the dead LOAD_VEC_A != 8 branch in the IQ4_XS shmem load; iq4_xs is in lut_load_vec_a()'s "8" list, so that path is never…
15 -
b11125
ggml-meta: resolve multi buffer views ( #29266 ) ggml-meta: resolve multi buffer views add TODO to revisit if graph allocator gets refactored Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49502431 macOS/iOS: macOS Apple Silicon…
20 -
b11124
cuda: top-k MoE should always fire ( #28432 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49495074 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
23 -
b11123
sycl : support new UT case for mul_mat_hadamard fp16 ( #29218 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49484593 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
4 -
b11122
sycl: extend MMVQ GLU fusion, add rms_norm+scale and ssm_conv+silu fusions ( #28931 ) sycl : extend MMVQ GLU fusion to mixed quant types; add rms_norm+scale and ssm_conv+silu fusions fixing spacing issue and macro converted to template function Website: https://llama.app…
7 -
b11121
sycl : support op get_rows_back, only support fp32/fp16 ( #25266 ) resovle confict support gedt_rows_back, update the ops.md Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49462636 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
15 -
b11120
vulkan: hide internal symbols to prevent duplicate-dlopen state destruction ( #29139 ) Since #28732 our internal symbols are exported. A duplicate copy dlopened and dlclosed by ggml_backend_load_all() then interposes them, so its destructors destroy the live vk_instance and…
25 -
b11119
sampler: reduce the size of the probe ( #29285 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49443624 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
33 -
b11118
hex-dma: introduce direct-mapped DMA cache that is better suited for HVX FA mask handling ( #29282 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49404271 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
6 -
b11117
HIP : optimize IQ2/IQ3 ( __vsub4 __vcmpne4 ) using SWAR ( #27962 ) HIP : use bit manipulation for __vcmpne4 HIP : use bit manipulation for __vsub4 Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49401157 macOS/iOS: macOS Apple Silicon…
11