llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 15d ago
b11246
chat : fix Muse Glimmer ignoring response_format json_schema with --jinja ( #29615 ) chat : fix Muse Glimmer ignoring response_format json_schema with --jinja Fixes #29613 chat : accept json fences and clean up chat : fix choice parenthesis Co-authored-by: Alde Rojas…
8 -
b11243
ci : update the oneAPI toolkit to 2026.1 ( #29273 ) ci : update the oneAPI toolkit to 2026.1 oneDNN is removed from Intel Deep Learning Essentials in 2026.0, so staying on the deep-learning-essentials path would silently lose oneDNN support when the toolkit version is updated.…
13 -
b11242
mtmd: fix GCC 15 stringop-overflow in decode_embd_batch ( #29607 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50949667 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
9 -
b11240
server : support typed content (vision/audio/video) input for /v1/embeddings endpoint ( #29556 ) server : support multimodal input for /v1/embeddings (Qwen3-VL-Embedding) Accept the OpenAI-style wrapped content array format for multimodal embedding requests. Each {"content":…
10 -
b11239
vulkan: include functional header ( #29597 ) Fixes the compile error: no template named 'function' in namespace 'std' Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50927029 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
21 -
b11238
models: pad on the left with ggml_pad_ext ( #29567 ) models: pad on the left with ggml_pad_ext The Parakeet, LFM2-Audio, Granite Speech and Gemma 4 audio encoders build a left padding as a right pad followed by a roll, and DFlash2 concatenates a zero filled block in front of the…
13 -
b11237
ggml-openvino: mark unaligned batch-stride views unsupported ( #29603 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50908876 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
29 -
b11236
batch: migrate speculative, mtmd and server to batch_ext ( #29385 ) adapt common add common_batch wip wip: spec cont common_speculative_process server_batch to use common_batch rm some stale calls Assisted-by: Claude Fable 5.1 migrate mtmd handle imrope, handle return val of…
4 -
b11235
common : fix HF cache paths on Windows ( #29475 ) Supersedes #29158 Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50839831 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
38 -
b11234
webgpu: Handle unaligned writes in ggml_backend_webgpu_buffer_set_tensor ( #29471 ) Fix: Handle unaligned writes in ggml_backend_webgpu_buffer_set_tensor Clang formatting Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50830975…
26 -
b11232
ggml-cpu: enable tiled flash attention for non-vector-multiple head dims on x86 ( #29423 ) ggml-cpu: enable tiled flash attention for non-vector-multiple head dims on x86 add AVX2 support for masked loading and storing in simd_gemm_ukernel_tail ggml-cpu: fix FA softcap handling…
11 -
b11229
HIP: fix template skip for DKQ > 256 mfma kernels ( #29559 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50767985 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
14 -
b11228
metal: support left and circular padding in GGML_OP_PAD ( #29561 ) metal: support left and circular padding in GGML_OP_PAD Align Metal with CPU, CUDA and Vulkan: shift the source coordinates by the left paddings, wrap them around with the same wrap_around when circular, and read…
12 -
b11227
context : do not re-reserve the scheduler when toggling causal_attn ( #28751 ) context : do not re-reserve the scheduler when toggling causal_attn llama_context::set_causal_attn() marks the scheduler to do a full re-reserve on every change of the flag. For vision inputs, this…
27 -
b11226
Enables Windows ARM64 build with MSVC cl.exe ( #28362 ) can reproduce the issue vlad sees fix fma issue drop volatile fix volatile runtime task add arm flag if needed fix hsum compile error fix syntax in quants strengthen sve probing make the syntax fixes one liners remove debug…
13 -
b11225
tests : fix ggml init ( #29554 ) tests : init ggml for test-recurrent-state-rollback cont : same for test-save-load-state cont : add to test-state-restore-fragmented + add TODOs Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50700507…
14 -
b11224
vulkan: fix wrong results when a mul_mat reads a slice of a larger cache ( #28956 ) vulkan: read the batch stride of an in place src0 from nb[2] A dim01 contiguous tensor can still be a view whose batches are strided by more than ne[1] rows, the first rows of a KV cache for…
25 -
b11223
server : allow RANK pooling batch splitting for causal LLM rerankers (ie. Qwen3 and Qwen3-VL) ( #28876 ) server : allow splitting RANK pooling for causal LLM rerankers Rerank models fall into two categories: bidirectional cross-encoders (BERT, etc.) that require all tokens in a…
26 -
b11222
common : avoid side effects around params parsing ( #29537 ) register --rpc unconditionally and call llama_supports_rpc() only from its handler print server "initialization ..." log after args are parsed Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL Website: https://llama.app…
32 -
b11221
common : make string_split throw on invalid values ( #29518 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50573055 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
32 -
b11218
jinja : add support for dict builtin ( #29477 ) add support for dict builtin add tests Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50563209 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
15 -
b11217
opencl: refine bin kernel loading condition ( #29503 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50560500 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
19 -
b11216
sycl: FWHT kernels for block widths above 512 ( #29243 ) The SYCL FWHT covers 64 to 512 via the standard butterfly network, plus 384/640/768/1280 via the Kronecker/Paley construction added separately in Hadamard hint can produce (1024, 2048, 4096, 8192); those still fall through…
17 -
b11215
CUDA: tune fp16 tile FlashAttention configs for head sizes 40-112 ( #26289 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50554486 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
28 -
b11214
HIP: Enable fattn-mma kernel on cdna for dkq > 256 for large batch sizes ( #28907 ) HIP: Enable fattn-mma kernel on cdna for dkq > 256 for large batch sizes CI: hip-quality-check: ignore spills for very large mfma mma kernels Website: https://llama.app Attestations:…
34 -
b11213
vulkan: fix argsort kernel selection for Adreno ( #29469 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50544436 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
21 -
b11212
common : throw instead of abort on grammar without llguidance ( #29516 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50541746 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
14 -
b11211
RPC: use RDMA completion channel to not spin ( #29440 ) RPC: use RDMA completion queue to not spin add TODO for apple RDMA Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50534798 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
25 -
b11209
llama-bench : fix OOB access of hf_file ( #29515 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50528155 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
8 -
b11208
hrm : fix layer placement of z_l_init weight ( #29512 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50522371 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
16 -
b11207
hexagon: support tiled Q4_0 and Q8_0 GET_ROWS ( #29511 ) hexagon: support tiled Q4_0 and Q8_0 GET_ROWS hex-get-rows: fix macros hex-get-rows: use tiled HVX dequantization Assisted-by: OpenCode hex-get-rows: fix register spills and clean up checks for unsupported ops…
24 -
b11206
hexagon: support for backend sampler ( #29502 ) hex-topk: trying to improve/cleanup the pipeline hex-sampling: add STEP op hex-sampler: add SUM op hex-sampler: update CPY to support sampling cases hex-binary: add support for chunking to handle large logits hex-argmax: super…
37 -
b11205
cuda: support Nemotron 3 Puzzle state size 96 for ssm scan ( #28717 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50461180 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
13 -
b11203
cuda: add F16 input to the FWHT ( #29096 ) cuda: add F16 input to the FWHT The CUDA FWHT accepts F32 input only. This makes the source type a template parameter, so the kernel reads an F16 source directly instead of requiring a converted copy. The F32 path is unchanged.…
33 -
b11202
server : fix wake_fd warning on Windows ( #29479 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50455005 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
19 -
b11201
Revert "Change max context length for auto-fitting with unified KV ( #28849 )" ( #29437 ) This reverts commit b04d4e5 . Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50433369 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
17 -
b11200
jinja : implement sameas test ( #29448 ) implement sameas test add tests Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50397675 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
17 -
b11199
jinja : fix compile error ( #29468 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50395469 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
36 -
b11195
ggml-cpu: tiled mul_mat for k-quants ( #27851 ) Added tiled mul_mat. For each mul_mat_one_chunk, quants are unpacked into (max) 256x256 tiles of int8, one routine per quent. Then microkernel computes 16x16 tiles before writing out 256x256 float reults to main memory.…
31 -
b11194
opencl: add A8 Q8_0 non-MoE dp4a binary kernel ( #29439 )
17 -
b11193
hexagon: find software divide calls using binary inspection tool ( #29449 ) hex-scripts: fix table alignment hex-scripts: find sw div calls using binary inspection tool Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50359675…
28 -
b11192
vendor : update cpp-httplib to 0.58.0 ( #29407 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50337762 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
22 -
b11191
common,rpc : simplify fs_create_directory_with_parents() ( #29432 ) The original function was broken on Windows for some unicode paths Paths without a trailing separator now create the last directory too, matching the function name. All current callers already include a trailing…
26 -
b11190
mtmd: fix mel preprocessor in LFM2 audio ( #29403 ) which resulted in different greedy transcripts for 4.5% of English and 6.5% of Japanese test utterances. In Japanese, some differences changed entire words. This change: uses log(x + 2^-24) instead of clamping to the log floor…
27 -
b11189
opencl: add bin kernel kernel_gemm_noshuffle_q5_k_f32_32b_trans_ila_a8_bin , kernel_gemm_noshuffle_q5_k_q8_1_dp4a_ila_a8_bin ( #29401 ) opencl: add A8 Q5_K non-MoE non dp4a + dp4a binary kernel opencl: fix s transpose - s only transposed for bin kernels Co-authored-by: Li He…
18 -
b11188
Fixing the vulkan build issue of legacy GLSLC version that has no cooperativeMatrix API support ( #29373 ) ( #29409 ) vulkan : fix build issue of legacy glslc version by adding GGML_VULKAN_COOPMAT_GLSLC_SUPPORT macro check for Intel FA shader compiling vulkan : add preprocess…
21 -
b11185
common : extract shared unicode path/string helpers ( #29415 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/50264913 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
22 -
b11184
metal: FWHT kernels for block widths above 512 ( #29095 ) metal: FWHT kernels for block widths above 512 The Metal FWHT covers widths 64 to 512, one row per simdgroup with N/32 values per lane. Wider blocks need more registers per lane than that layout allows. kernel_fwht_tg…
28