llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 25d ago
b11047
cuda : fix CUB argsort corruption caused by in-place keys ( #28389 ) argsort_f32_i32_cuda_cub called the one-shot DeviceRadixSort::SortPairs API with d_keys_in == d_keys_out (temp_keys, temp_keys). CUB's internal double-buffer ping-pong requires distinct key buffers: with…
22 -
b11046
opencl: add support for bin kernel flash_attn_f32_f16_bin ( #29046 ) opencl: add flash_attn_f32_f16_bin opencl: guarded prefill fa Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48577094 macOS/iOS: macOS Apple Silicon (arm64) macOS…
28 -
b11045
hexagon: add ROLL op support ( #29105 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48565469 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
18 -
b11044
hexagon: im2col update ( #29103 ) ggml-hexagon: accept 1D and padded IM2COL ops ggml-hexagon: make pure-DDR IM2COL kernel is_2D-aware ggml-hexagon: extend IM2COL DMA patch-embed fast path to 1D ggml-hexagon: add blocked-staging general IM2COL DMA kernel Website:…
9 -
b11043
hexagon: HMX flash-attention head_dim padding (support DK=DV=72) ( #26539 ) Allow HMX flash-attention to run with head_dim not a multiple of 64 (e.g. SigLIP head_dim=72), by operating on DK/DV rounded up to 64 with zero-filled tail lanes. Website: https://llama.app Attestations:…
29 -
b11042
opencl: add bin kernel kernel_gemm_noshuffle_q6_k_f32_32b_trans_ila_a8_bin ( #28678 ) opencl: add A8 Q6_K non-MoE binary kernel opencl: fix layout compatibility Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48517412 macOS/iOS: macOS…
14 -
b11040
ggml : check for allocation failures to prevent crashes ( #28149 ) ggml : check for allocation failures to prevent crashes wording Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48473967 macOS/iOS: macOS Apple Silicon (arm64) macOS…
17 -
b11039
Model-Saver: Write the SWA pattern, 15 more architectures roundtrip ( #29042 ) llama: read the SWA pattern as a period or a per-layer array Add llama_model_base::load_swa_pattern(), which reads sliding_window_pattern either as one flag per layer or as a period expanded by…
7 -
b11037
ggml-webgpu: fix supports_op condition for GET_ROWS ( #28978 ) fix get_rows vec4 handling Add src strides checking to vec4_aligned of get_rows and the new test case. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48438627 macOS/iOS:…
31 -
b11036
ggml : handle graph buffer reservation failure ( #26070 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48431150 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
20 -
b11034
vocab : add ufakzeka pre-tokenizer ( #29033 ) vocab : add ufakzeka pre-tokenizer vocab : move ufakzeka to the models list and regenerate the hash mapping Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48421591 macOS/iOS: macOS Apple…
6 -
b11030
ci : bump android-actions/setup-android to 4.0.4 ( #29065 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48395907 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
37 -
b11029
vulkan: raise the hoisted row-id limit for mul_mat_id from 256 to 1024 experts ( #28501 ) vulkan: raise the hoisted row-id limit for mul_mat_id to 512 experts The expert-count shader (count_experts.comp) sizes its shared arrays with BLOCK_SIZE, which is 256. Because of that,…
16 -
b11028
ci : add missing evict-old-files ( #29041 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48252257 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
5 -
b11027
rpc : skip ACCEL devices ( #29020 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48232585 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
15 -
b11026
model : skip gate_up_exps if TENSOR_SKIP is set ( #29014 ) required for qwen35moe if MTP tensors are fused but not loaded Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48194938 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
5 -
b11025
model : extend Nemotron MTP support ( #29018 ) first fix removed unnecessary declarations Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48188464 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
14 -
b11024
openvino : Update OpenVINO to 2026.4;fix clangd,MSVC warnings; ( #29009 ) Update to openvino-2026.4 Update OV docs ggml-openvino : fix clangd and MSVC warnings fix int to ptr cast, more internal linkage enforcement, and avoiding duplicate switch case Co-authored-by: Mostafa…
8 -
b11022
vulkan: split buffers and debug code into separate files, add shared headers ( #28732 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48170206 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
33 -
b11020
chat : add message delimiters to the DeepSeek V3.2/V4 parser ( #29008 ) chat : add message delimiters to the DeepSeek V3.2/V4 parser Assisted-by: Claude Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Website: https://llama.app Attestations:…
8 -
b11019
gguf : align the data section relative to the GGUF start, not the file ( #28993 ) gguf : align the data section relative to the GGUF start, not the file gguf_init_from_file_ptr reads a GGUF from the current file position, but padded the data section from file offset 0, so a GGUF…
9 -
b11018
sycl : fix the B70 mem allocate error when >19.3GB ( #28953 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48150237 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
35 -
b11017
vulkan: skip unneeded MoE work in mul_mm coopmat1 path ( #25483 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48144237 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
25 -
b11016
sycl: ssm_conv: fuse the SiLU epilogue into the ssm_conv kernel ( #28929 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48138242 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
38 -
b11015
opencl: fix various warnings ( #28984 ) opencl: fix warnings opencl: fix warnings for non adreno Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48132480 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
11 -
b11013
vulkan: fix buffer_reference alignment in im2col shaders ( #28996 ) Both im2col.comp and im2col_3d.comp declare D_ptr without an explicit buffer_reference_align, so glslang emits writes through it as Aligned 16. The shaders advance the pointer by D_SIZE, a per-variant define set…
33 -
b11012
vulkan: support qwen4exp hc ops ( #28988 ) vulkan: support qwen4exp hc ops fix stale comment [no-ci] Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48109146 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
30 -
b11011
[SYCL] Fix function signature for ggml_backend_sycl_split_buffer_type ( #28981 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48082811 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
36 -
b11010
vulkan: work around NV bug with argsort_large.comp ( #28975 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48059150 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
23 -
b11009
TP: fix split state and granularity for fused QKV gemma4, qwen35 ( #28965 ) model: calculate split states for attn_qkv from n_head * n_embd_head_k required for gemma4 with --fuse-qkv, where n_embd is 5376 but Q is 8192. model: handle fused full attention layers for…
9 -
b11007
Enable CUDA graph for MTP draft ( #28549 ) Improve CUDA graph usage for MTP Rename field Address review feedback Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47982528 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
13 -
b11006
hexagon: Support for K-Quants Q4_K and Q6_K ( #28994 ) implement q6k/q4k kernels Squashed from: feat: implement q6k kernel hex-q6k: improve unpack accuracy hex-q4_k: add support for Q4_K kernels Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com Website: https://llama.app…
37 -
b11005
hexagon: accept the zeroed rope probe in supports_op ( #28995 ) llama probes weight placement with a rope where all params are 0, so rejecting n_dims == 0 or freq_base == 0 puts rope_freqs on the CPU. That splits the decode graph at every full-attention layer (gemma-4-E2B: 5…
21 -
b11003
model : add support for HrmTextForCausalLM (DFM Mimir 1B) ( #27625 ) model : add support for HrmTextForCausalLM (DFM Mimir 1B) HRM-Text runs two transformer stacks (low, high) in an alternating cycle over the same token stream. The low-cycle state z_l starts from a learned…
11 -
b11002
CUDA/HIP: improve access patterns in im2col ( #28013 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47937267 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
10 -
b11001
spacemit : fix wrong transpose function for int16 data ( #25161 ) The sizeof(int16_t) branch in permute_transpose_impl calls rvv_transposed_s32_mn_to_nm instead of rvv_transposed_s16_mn_to_nm . This is a copy-paste bug from the sizeof(int32_t) branch above it. The s32 function…
24 -
b11000
rpc : invalidate cached compute graph when a referenced buffer is freed ( #24292 ) The server caches the most recent compute graph per device so that GRAPH_RECOMPUTE can re-execute it without resending tensor data. The cached graph nodes hold direct pointers to backend buffers…
9 -
b10999
Change max context length for auto-fitting with unified KV ( #28849 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47906437 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
26 -
b10998
qwen4exp: add hc ops ( #28901 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47897353 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
10 -
b10997
HIP: broaden MoE ncols_opt tile heuristic on RDNA3.5 architecture ( #28935 ) It's found the MoE ncols_opt tile heuristic needs to be broadened to include the RDNA3.5 architecture. The code change is implemented in ggml/src/ggml-cuda/mmq.cu and just change the…
14 -
b10996
chat : force \n</think> on reasoning budget end for qwen3-coder ( #28869 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47858837 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
29 -
b10995
vulkan: make MUL_MAT_ID BN/2 tail unconditional ( #28923 ) Use BN/2 as the default for BNover2 and as the disabled fallback for BNover4, and remove the enable gate from the MUL_MAT_ID BN/2 branch. The BN/4 branch remains gated by enable_smaller_matrices, while the p.N path is…
25 -
b10994
metal: fix NaN in mul_mm_id when activations exceed f16 range ( #26223 ) test-backend-ops: reproduce MUL_MAT_ID NaN for activations beyond f16 The Metal mul_mm_id path narrows src1 to half for the simdgroup MMA ( S1 = half in every instantiation; ggml-metal.metal:10582 and…
28 -
b10993
ci : add self-hosted webgpu to hf-jobs ( #28712 ) add self-hosted vulkan and webgpu to hf-jobs try t4-medium cont : adjust cpu backend threads try t4-small again restore cm jobs Co-authored-by: Georgi Gerganov ggerganov@gmail.com Website: https://llama.app Attestations:…
29 -
b10992
llama-bench: support --version to print build info ( #28971 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47831792 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
28 -
b10991
hexagon: add back missing contiguous fast-path and hvx_copy_uu for each run ( #28886 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47775478 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
15 -
b10990
hex-cpy: use dma if src and dst are contiguous ( #28906 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47770889 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
34 -
b10989
HIP: Enable AllReduce for ROCm ( #27825 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/47762387 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
30