llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 10d ago
b11386
common : prepare load_from_models_dir() for path conversion ( #29674 ) This is part of the fs::path modernization series. That was also the opportunity to remove fs_list(). Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations:…
17 -
b11387: spec : fix n-gram drafts rejected at temp > 0 after truncation (#29924)
Co-authored-by: Pranesh Gonegandla pgonegandla@nvidia.com
7 -
b11385
server : fix dead LLAMA_ARG_HF_REPO_FILE key in preset allow-list ( #2 …
33 -
b11382
webgpu: add f16 support to fill/set_rows ( #29897 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52497832 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
15 -
b11381
mtmd : fix deprecated strdup warning on Windows ( #29863 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52479259 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
29 -
b11380
vendor : update cpp-httplib to 0.59.0 ( #29886 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52476117 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
23 -
b11378
common : add common_is_tty() helper and fix deprecated warnings on Windows ( #29860 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52429962 macOS/iOS: macOS Apple Silicon (arm64)…
15 -
b11377
chat : honor json_schema in Ling 3.0 parser ( #29813 ) chat : honor json_schema in Ling 3.0 parser Ling 3.0 only built a grammar for tool calls and did not handle inputs.json_schema, so response_format requests were left unconstrained. Add an eager response-format grammar path…
27 -
b11376
ci: fix flaky ADD_ADD f16 by using the fused ADD tolerance ( #29904 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52418791 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
5 -
b11375
graph: gather the recurrent states once so the reserve covers every split ( #29856 ) build_rs gathered the extra states (n_rs - n_seqs rows) with their own get_rows. The worst-case reserve has n_rs == n_seqs, so that node was sized at zero rows, and any ubatch whose cells are…
16 -
b11374
ggml-openvino: update to 2026.4.1, optimize performance, expand ops, improve device listing. ( #29852 ) ggml-openvino : Qwen3.5 MoE perf ( #312 ) Squash of ravi9#312 : ggml-openvino: add detailed inference profiling (Yu, Zijun) ggml-openvino: use remote output tensors by default…
16 -
b11372
qwen4exp : halve the indexer score memory ( #29825 ) qwen4exp : halve the indexer score memory The indexer scored all heads in one product and rectified a copy of it, so two [n_pool, n_idx_h, n_tokens] f32 tensors were live at once, the largest buffers of the graph at long…
25 -
b11370
CUDA: fuse shared experts into MMVQ ( #29184 ) CUDA: fuse shared experts into MMVQ check if buffer is null move stride_col_dst to fusion args Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52371643 macOS/iOS: macOS Apple Silicon…
36 -
b11368
spec : add probabilistic sampling for simple draft and MTP ( #27694 ) Make the drafter probabilistic and the target verify by rejection sampling Drop stale spec_draft_q before drafting Fallback to argmax sampling for grammar-constrained requests and adding flag for enabling…
28 -
b11366
ggml-quants : avoid invalid rounding in qkx3 scale search ( #29817 ) ggml-quants : avoid invalid rounding in qkx3 scale search The imatrix scale search can produce an infinite, NaN, or otherwise out-of-range value when the fitted minimum collapses to the maximum or makes the…
31 -
b11365
ggml-cpu : fix soft_max_back wrong output when dst aliases src1 ( #27096 ) ggml-cpu : fix soft_max_back wrong output when dst aliases src1 GGML_OP_SOFT_MAX_BACK is listed in ggml_op_can_inplace, so the graph allocator may assign dst to alias either src0 (dy) or src1 (y). The…
10 -
b11364
model: support nimble decision model ( #29844 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52332773 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
18 -
b11371: model: add support for clef decision model (text-only) (#29831)
init support for clef (text only) more static graph clean up nits nits 2 Update gguf-py/gguf/constants.py Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co
21 -
b11362
metal : add tensor API flash attention kernel for F16 KV ( #29570 ) metal : add tensor API flash attention kernel for F16 KV cont : add tensor FA kernels for DK=DV=512 and DK=576, DV=512 cont : support attention sinks, ALiBi and logit softcap in the tensor FA kernel cont : add…
19 -
b11361
llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) ( #29818 ) init conversion convert: ok model loaded add server code improve conversion script support shared prompt prefix add docs, imorove UX a bit add vision support add openjev tiny model for…
21 -
b11355
vulkan: disable large matmul tile on Samsung GPUs with 32KB shared memory ( #28531 ) Assisted-by: Claude Opus Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52313922 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
13 -
b11352
qwen4exp : optimize mask constructions ( #29824 ) qwen4exp : optimize mask constructions cont : apply the same change for GLM5-next Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52303821 macOS/iOS: macOS Apple Silicon (arm64) macOS…
31 -
b11351
ggml : add alloc_buffer_n to buffer type interface ( #23671 ) ggml : add alloc_buffer_n to buffer type interface Add alloc_buffer_n method to ggml_backend_buffer_type_i interface, with a public API ggml_backend_buft_alloc_buffer_n. Default implementation in ggml-backend.cpp…
20 -
b11349
vulkan: add logging to pipeline compile issues ( #29794 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52203370 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
38 -
b11346
qwen4exp: fix tests ( #29819 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52169567 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
32 -
b11344
CUDA: fix 2 broken Volta FA cases ( #29803 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52093363 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
9 -
b11339
llama : clamp kpool re-pool bound to existing pools ( #29805 ) tests : simplify function signature llama : clamp kpool re-pool bound to existing pools The n_tokens/kpool + n_seqs_unq bound on n_new_g overshoots when a batch fills the whole cache: n_ctx tokens complete exactly…
22 -
b11338
hexagon: shared strided DMA copy for CPY and CONCAT, any-dim CONCAT via DMA ( #29685 ) hexagon: shared strided DMA copy for CPY and CONCAT, any-dim CONCAT via DMA hex-cpy: various fixes on top of the concat optimizations Removed CONCAT_DMA_MIN_ROW logic, it was broken with…
33 -
b11337
server: return HTTP 400 for invalid embedding requests ( #29060 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52037093 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
9 -
b11335
cuda : route sm70 to the Turing MMVQ nwarps table ( #29753 ) cuda : route sm70 to the Turing MMVQ nwarps table Volta (sm_70) has no MMVQ parameter table of its own and falls through to GENERIC, which launches K-quant batch-1 decode (ncols_dst == 1) at nwarps=4. sm_70 shares…
5 -
b11334
metal : release temporary private transfer buffers ( #29777 ) metal : release temporary private transfer buffers Assisted-by: OpenAI Codex metal : fix order and formatting Co-authored-by: Niklas Wenzel dev@nikwen.de Website: https://llama.app Attestations:…
19 -
b11333
webgpu: add bfloat16 support for MUL_MAT/MUL_MAT_ID/GET_ROWS- #29358 ( #29358 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52019658 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
15 -
b11332
llama : fix invalid assert in recurrent memory ( #29799 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52014664 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
22 -
b11331
CUDA: Handle compute type for NVFP4 on cublass path ( #29173 ) CUDA: Handle compute type for NVFP4 on cublass path Signed-off-by: ynankani ynankani@nvidia.com Use BF16 compute type for quantized models if HW allows Signed-off-by: ynankani ynankani@nvidia.com Set acc prec to bf16…
26 -
b11330
Qwen4Exp: add MTP ( #29761 ) Qwen4Exp: add MTP remove has_state member, check via ctx_bufs being non-empty consistent naming + less verbose comments cont : clean-up recurrent memory cont : clean-up comments Co-authored-by: Georgi Gerganov ggerganov@gmail.com Website:…
28 -
b11327
mtmd: cap max_image to ubatch for non_causal models ( #29773 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51957111 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
30 -
b11326
meta: clear inactive AllReduce shards with FILL, not SCALE ( #29793 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51949614 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
14 -
b11325
jinja : skip copying loop scope unless a loop filter needs it ( #29776 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51940374 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
37 -
b11324
llama-mmap : avoid a second full-size copy of each tensor with direct-io ( #29749 ) Assisted-by: Claude Co-authored-by: Pranesh Gonegandla pgonegandla@nvidia.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51929570 macOS/iOS: macOS…
5 -
b11323
HIP: avoid treating CDNA as dgx spark for gqa_ratio 20 in fattn_mma dqk 576 ( #29572 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51919562 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
30 -
b11322
hex-workqueue: fix race condition in seqn getting out of sync with idx_read/write ( #29785 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51906249 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
16 -
b11321
BLAS : Document AOCL-BLAS build and label the device AOCL-BLAS ( #29640 ) BLAS : Document AOCL-BLAS build and label the device AOCL-BLAS AOCL-Blas : Add an AOCL-BLAS Quick Start and drop the fixed version path AOCL-BLAS doc : Note on ZenDNN Website: https://llama.app…
6 -
b11320
common : add LLM-jp-4.1 Harmony dialect handler ( #29681 ) LLM-jp-4.1 uses the GPT-OSS format, but its tokenizer decodes a space after every special token and parallel tool calls are separated by <|end|>. The GPT-OSS handler rejects this output, so add a dedicated handler,…
21 -
b11319
opencl: mark vec subgroup bcast as supproted for Adreno E17 compiler ( #29698 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51780568 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
10 -
b11318
vocab : honor BOS/EOS settings for PLaMo-2 and PLaMo-3 ( #29734 ) vocab : honor BOS/EOS settings for PLaMo-2 and PLaMo-3 The original tokenizer configs for PLaMo-2 and PLaMo-3 have add_bos_token: true and add_eos_token: false , but _set_vocab_plamo() did not write the BOS/EOS…
19 -
b11317
llama-bench : fix verbosity filter to show GGML_LOG_ERROR ( #28229 ) bench : fix verbosity filter to show GGML_LOG_ERROR ( #28107 ) bench: remove dead variables Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/51761023 macOS/iOS: macOS…
36