llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 13d ago
b11316
metal : use bf16 math for mxfp4 mul-mat ( #29770 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51752863 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
20 -
b11313
model : re-enable -sm tensor for qwen4exp ( #28569 ) #27941 disabled -sm tensor for qwen4exp because test-llama-archs asserted on the Meta device once the fixture carried a PLE layer: GGML_ASSERT(ggml_backend_buffer_is_meta(tensor->buffer)) at ggml-backend-meta.cpp:476. With…
37 -
b11312
webgpu: fix SSM_SCAN binding aliasing ( #29750 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51685637 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
24 -
b11311
ggml-opencl : replace alloca() with std::vector ( #29765 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51681384 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
5 -
b11310
cuda: guard the iq4_nl dequantize row kernel against short rows ( #29683 ) dequantize_block_iq4_nl writes QK_K values per block, but a row can be shorter than that (an IQ4_NL row is only guaranteed to be a multiple of QK4_NL). Threads whose 32-value sub-block starts at or past k…
4 -
b11309
Hexagon: optimize ALLREDUCE with support for safe scatter mode ( #29757 ) hex-allreduce: add support for safe scatter mode hex-allreduce: pare down excessive comments hex-allreduce: re-write to remove register spills Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com Website:…
25 -
b11308
args: fix cli download mmproj arg ( #28977 ) tests: add tests for cli download arg parsing args: fix cli download mmproj arg Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51664692 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
8 -
b11307
llama : preserve original batch order for speculative decoding layer inputs ( #29019 ) llama: preserve original batch order for layer inputs Assisted-by: Codex tests: cover layer-input order across KV layouts Assisted-by: Codex tests: exercise layer-input ordering on CUDA…
23 -
b11306
test-llama-archs : toggle causal_attn to catch graph shape changes ( #29724 ) After the device decode, flip causal_attn off, decode n_ubatch/2 then n_ubatch tokens. Both have the same node count, so a shape that depends on the flag makes the second reallocate at an unchanged…
26 -
b11304
llama: properly handle KV on training ( #28520 ) llama: properly handle KV on training improve Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51644581 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
29 -
b11303
batch: migrate the rest of examples to llama_batch_ext ( #29601 ) migrate the rest test-thread-safety rm common_batch_staged Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51638011 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
25 -
b11302
glm5-next: give dead indexer slots unique scatter rows ( #29745 ) The sparse indexer mask is built with a set_rows scatter. Padded pools, absent sequences and missing tail cells all pointed to the same n_kv sentinel row, and invisible pools picked by top_k to fill the selection…
13 -
b11299
cli: exit on stdin EOF and drop the console wide Ctrl+C broadcast ( #29722 ) cli: exit on stdin EOF and drop the console wide Ctrl+C broadcast On Windows the simple input reader sends CTRL_C_EVENT to every process attached to the console when stdin reaches EOF, killing unrelated…
13 -
b11298
mimo : support dflash (convert + feature extraction) ( #29650 ) convert : update to support dflash cont : fix Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Website: https://llama.app…
37 -
b11297
jinja : support coerced array attributes ( #29574 ) support coerced array attributes add tests Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51590476 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
13 -
b11295
ci : fix Models Backend Check by shortening the hrm_text fixture ( #29744 ) The fixture recycles its two blocks over 8 cache slots, so the fp16 error builds up past the 1e-4 NMSE bound on the Vulkan T4 and WebGPU jobs of Models Backend. Two l-cycles keep every branch of the…
9 -
b11294
llama: llama_prefetch_rows ( #29599 ) llama: llama_prefetch_rows llama: support row prefetch on Windows Apply the Windows port contributed by @praneshgo unchanged. Source: #29599 (comment) avoid exposing llama-mmap in model code, route via llama-impl add windows check, only…
31 -
b11293
ggml : add BF16 unary, GLU, binary and scale ops (CPU, CUDA) ( #29675 ) ggml : add BF16 unary, GLU, binary and scale ops (CPU, CUDA) ggml-cpu : use per-op _bf16 functions for BF16 unary and GLU ops Assisted-by: Claude Opus 5.5 CUDA: use ggml_cuda_cast in binbcast and unary…
38 -
b11292
cpu: accept BF16 in src1 of mul_mat ( #28937 ) cpu: accept BF16 in src1 of mul_mat ggml_conv_1d_dw builds its im2col in F32 when the kernel is BF16, then calls ggml_mul_mat(im2col, kernel), which puts F32 in src0 and BF16 in src1. The CPU backend refused that combination, so it…
9 -
b11284
openvino: serve GET_ROWS on a weight view from the base Constant ( #28381 ) openvino: serve GET_ROWS on a weight view from the base Constant Resolve view_src when collecting weight Constants so a view over a quantized weight no longer becomes a dynamic typed Parameter, and fold…
20 -
b11282
musa : define CUDA_ARCH for device passes ( #29508 ) The MUSA vendor header never defined CUDA_ARCH , so every architecture test in the shared ggml-cuda sources evaluated to 0. Kernel bodies gated on the architecture therefore compiled to nothing, for example the q8_0 -> f16…
20 -
b11280
SYCL: reduce tensor allreduce sync with pinned host buffers ( #29604 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51446793 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
6 -
b11279
add GLM-5.3-Flash (GLM5-Next) support ( #27773 ) Rebase GLM-Next support onto master, and migrate to llama-memory-hybrid-idx Add initial MTP support Merge branch optimizations. Reduce allocated compute buffer size, speed up long context decode, fla, and slight MTP improvements.…
6 -
b11278
vendor: update BoringSSL to 0.20260929.0 ( #29669 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51432581 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
19 -
b11276
Hexagon f16 activation ops ( #29209 ) hexagon: add F16 support for activation ops (SILU/GELU/GELU_QUICK/GEGLU/SWIGLU) Widens ggml_hexagon_supported_activations() to accept F16 (src0/dst/src1 must agree on type), and adds F16 per-thread worker functions in act-ops.c mirroring the…
35 -
b11275
gguf : reject tensor size that wraps after padding ( #26979 ) GGML_PAD(nbytes, alignment) wraps to 0 when nbytes is within (alignment - 1) of SIZE_MAX, which silently bypassed the size overflow guard in gguf_init_from_reader. Reject the tensor before padding when nbytes +…
33 -
b11274
ggml : check row bounds in get_rows_back ( #29575 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51392411 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
12 -
b11273
model : support classifier_pooling for rerankers ( #29627 ) model : support classifier_pooling for ModernBERT rerankers Assisted-by: Claude Opus 5.5 model : read classifier pooling type in load_hparams Write classifier.pooling_type from _try_set_pooling_type whenever the config…
20 -
b11272
hexagon: optimize concat op ( #29673 ) hex-concat: reduce pkts in gather/transpose hot loop gather directly into dst buffer, use special instruction for gather sync hex-concat: use fastdiv replace calls to sw divide with fastpath hex-concat: optimize DMA-HVX pipeline and add…
28 -
b11277: ggml-zdnn: impl buffer reset, fix memory leaks (#29637)
cont: fix code style Signed-off-by: Aaron Teo aaron.teo1@ibm.com
24 -
b11270
ggml-cuda: HIP: optimize packed byte subtraction ( __vsubss4 -> __vsub4 ) ( #29478 ) ggml-cuda: HIP: optimize non-saturating packed byte subtraction ( __vsubss4 ) CI: ignore 1 spilled vgpr in fattn_vec Co-authored-by: Carl Philipp Klemm carl@uvos.xyz Website: https://llama.app…
14 -
b11269
ci: add zdnn backend build but not test ( #29541 ) ci: add zdnn backend build but not test Signed-off-by: Aaron Teo aaron.teo1@ibm.com ci: attempt to run a ubuntu 26.04 container Signed-off-by: Aaron Teo aaron.teo1@ibm.com ci: set shell to bash Signed-off-by: Aaron Teo…
37 -
b11268
opencl: fix get_tensor for q5_K adreno gemm_nonshuffle kernel ( #29555 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51311810 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
34 -
b11267
vulkan: Tune GDN kernel, fix Intel performance ( #29476 ) vulkan: tune GDN shader tune for Intel Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51306598 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
8 -
b11266
vulkan : Load F32 A matrix 2 at a time when its 2-aligned ( #29254 ) It turns out Intel doesn't particularly like loading F32s one at a time and we already have the _2aliagned load logic in mul_mat_vec, so here we use it. While we do already check all the requirements to load…
6 -
b11265
vulkan: MOE aware mat_mul_id tile selection ( #29182 ) mut_mul_id selected its matmul tile with total token count. For MoE dispatch grid the true N per workgroup is per-expert rows. At pp128 on Sarvam 30B that is 6, not 128, so the picker took the l-tile for ~6 live rows. Most…
15 -
b11264
ggml : fix c++ odr by properly using GGML_COMMON_DECL_CPP ( #29504 ) fix c++ odr by properly using GGML_COMMON_DECL_CPP using actual field rather than macro Co-authored-by: Georgi Gerganov ggerganov@gmail.com Co-authored-by: XZiar xziar@xziar.xziar Co-authored-by: Georgi…
21 -
b11263
vocab : keep NORMAL in PLaMo-2 and PLaMo-3 ( #29580 ) vocab : keep NORMAL in PLaMo-2 and PLaMo-3 The PLaMo-2 and PLaMo-3 vocabularies mark as NORMAL. Current EOG token heuristic matched it by text and added its attribute to CONTROL. Skip this heuristic for the PLAMO2 vocab type…
37 -
b11262
ggml : accumulate f16 dot products in f32 on AVX512-FP16 ( #29545 ) Supersedes #29530 Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51274261 macOS/iOS: macOS Apple Silicon (arm64)…
19 -
b11261
ggml : require input tensors to be GGML_OP_NONE ( #29647 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51266318 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
16 -
b11260
hexagon: add FP32 GELU_ERF and GEGLU_ERF support ( #29631 ) hexagon: add FP32 GELU_ERF and GEGLU_ERF support hex-erf: reduce register pressure in kernels Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51243904 macOS/iOS: macOS Apple…
5 -
b11259
common : stop accepting draft tokens at EOG ( #29638 ) common : stop accepting draft tokens at EOG cont : remove the test Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51199980 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
23 -
b11258
server : remove the built-in UI's service worker when the UI is not served ( #29565 ) With --path or --no-ui, /sw.js returned 404, and a 404 does not remove a service worker, so browsers kept showing the cached built-in UI. Serve a worker that unregisters itself, clears its…
10 -
b11257
common : use fs::path for config dir ( #29649 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51176378 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
18 -
b11256
tests : adjust server string regex to also match m2 utlra results ( #29648 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51122498 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
37 -
b11255
common : add fs_write_atomic() ( #29642 ) Check for buffered write errors when closing downloaded files. Use UTF-8 paths when writing ETag files on Windows. Write in binary mode on Windows. Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app…
13 -
b11254
ggml : collect all input tensors into graph_inputs ( #29634 ) graph_inputs was populated while splitting the graph, so it only contained the inputs that are used as srcs of some node. With pipeline parallelism (n_copies > 1), each graph input contributes n_copies leafs to…
25 -
b11249
llama : fix init in several tools/examples ( #29632 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51066139 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
26