llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 3d ago
b11558
opencl: fix image limit for q4_k dense bin kernels, q4_0 bcast and rm…
19 -
b11556
model : support for Prism Bonsai 2 27B ( #29600 ) Runtime support for Prism Bonsai 2 27B Assisted-by: Claude Code address prism hadamard runtime feedback move hadamard tensors into method, define folded weight load_* hadamard fixes cont : clean-up conversion cleanup Assisted-by:…
34 -
b11554
vulkan: handle mul_mat_id duplicates in the prepass rather than looping ( #29998 ) vulkan: compute every row of an expert in mul_mm_id when ids repeat vulkan: handle mul_mat_id duplicates in the prepass rather than looping Extend the "hoist row ids" optimization to always be…
4 -
b11555
cpu: fix AVX2 CPUs not using F16C when available ( #30270 )
12 -
b11553: server: add component tests (#30312)
server: add component tests nits add codeowners
13 -
b11552
server: leave a busy slot untouched when a request pins it ( #30295 ) A request asking for a busy id_slot still ran the prompt cache update on that slot before being deferred. When the RAM cache held a better match, it was loaded into the slot while another request was still…
22 -
b11551: model : support MiniCPM-V 4.7 (#29416)
mtmd : add MiniCPM-V 4.7 support Signed-off-by: tc-mb tianchi_cai@icloud.com model : allow mrope time from an extra position slot Signed-off-by: tc-mb tianchi_cai@icloud.com Update conversion/minicpm.py Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Slim down…
30 -
b11550
cmake(s390x): disable z17 target for unsupported compilers ( #30297 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54655455 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
20 -
b11548: spec: properly handle mtmd input for mtp (#30257)
spec: properly handle mtmd input for mtp nits
20 -
b11546
cuda: always use MMVQ for MUL_MAT_ID on sm_60 ( #27828 )
18 -
b11545
CUDA/HIP: fix race in flash_attn_ext_f16_process_tile when nbatch_com…
24 -
b11542: ggml: fix s390x all cpu build (#30140)
Signed-off-by: Aaron Teo aaron.teo1@ibm.com
8 -
b11541
opencl: add bin kernels kernel_gemm_moe_q4_k_q8_1_dp4a_bin, kernel_ge…
25 -
b11540
sycl : accelerate MXFP4 MoE with arithmetic decoding and weight reord…
28 -
b11539
vendor: apply deep nested json patch from upstream ( #30253 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54488539 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
34 -
b11537
graph : reorder get_rows for embeddings ( #30160 ) graph : reorder get_rows for embeddings cont : fix gemma4 and improve input embedding construction logic cont : add TODO for lora cont : fix raw embeddings path gemma4 : avoid ple cast in embeddings path Website:…
31 -
b11535
llama-bench: respect -fitc if bigger than required benchmark size ( #28331 ) Assisted-By: opencode,llama.cpp,Qwen3.6-35B-A3B,Qwen3.8-27B Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54430110 macOS/iOS: macOS Apple Silicon (arm64)…
30 -
b11538
CUDA: fix round issue, under MSVC the CPU and GPU agree ( #30229 )
31 -
b11533
opencl: fix kernel compilation for a6x GPUs ( #30176 ) opencl: skip kernel_cpy_f32_f32_pack on A6X to avoid shader compiler crash The A6x compiler backend found in iot device with a623 (E031.50.31.01) cannot handle kernels with a large number of arguments. Skip this kernel for…
4 -
b11536: ui: Models Manager Follow-up Improvements (#30228)
common : read a GGUF's trained context from its metadata common_get_gguf_n_ctx_train opens only the file's metadata (no_alloc, like common_get_decision_type) and reads .context_length, so a caller can learn the trained context without loading the model. It accepts both u32 and…
6 -
b11530
llama: keep the backend sampling graph static across ubatches ( #30223 ) The reserve builds n_outputs_max_per_seq sampling chains per sampler, while a decode built one per output row, so the graph changed its topology after the reserve and GGML_SCHED_NO_REALLOC builds aborted on…
24 -
b11532: model : use exact GELU for ModernBERT encoders (#30108)
model : use exact GELU for ModernBERT encoders Assisted-by: Codex model : keep tanh GELU aliases on ggml_geglu Assisted-by: Claude Opus 5.5 model : map gelu_python to ggml_geglu_erf Assisted-by: Claude Opus 5.5
18 -
b11528
meta : handle host views ( #30217 ) meta : handle views of tensors allocated on the host A view shares the memory of its view_src, so ggml-alloc never allocates a view in the buffer of the split it lands in - the scheduler copies the source into the split and the ops that use…
21 -
b11526
vulkan: fix rms_norm workgroup count overflow ( #30145 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54337638 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
20 -
b11527
webgpu: use 2D workgroup dispatch for all the ops which use 1D dispat…
33 -
b11523
ci: fix Models Backend webgpu by supporting GGML_OP_DUP ( #30216 ) The mixed batch path of PR-29622 writes the token rows with set_rows into a dup of the embeddings. WebGPU did not support DUP, so the dup ran on the CPU while the set_rows writing into it was scheduled on WebGPU,…
32 -
b11524: musa: drop mp_21 from the default architectures (#30203)
musa: drop mp_21 from the default architectures Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Address review comments Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Address review comments Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Signed-off-by: Xiaodong Ye…
36 -
b11521
server: use port 9931 by default ( #30159 ) server: use port 9931 by default revert unrelated changes Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54269123 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
27 -
b11518
metal: add the 128/96 flash attention kernels, accept every compiled …
26 -
b11517: CUDA: pass src1 precision to host MMQ config helpers (#30168)
CUDA: pass src1 precision to host MMQ config helpers Signed-off-by: ynankani ynankani@nvidia.com make src1 prec explicit Signed-off-by: ynankani ynankani@nvidia.com Signed-off-by: ynankani ynankani@nvidia.com
31 -
b11516: hexagon: fix IM2COL patch-embed DMA ring overflow (#30189)
hexagon: fix IM2COL patch-embed DMA ring overflow The exact-tiling (stride == kernel, no pad/dilation) IM2COL DMA kernel issues IC KH DDR->VTCM descriptors per output row without checking the return value of dma_queue_push(), and then pops IC KH times. The per-thread DMA ring…
37 -
b11515
sycl : Q5_K reorder-layout MMVQ and fused GLU ( #29375 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54173692 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
38 -
b11512
model : fix DFlash output head sharing ( #30111 ) llama : fix DFlash output head sharing Assisted-by: Codex dflash : read tied output weights from GGUF metadata Assisted-by: Codex llama : share tied word embedding metadata Assisted-by: Codex llama : remove DFlash embedding head…
23 -
b11513: CUDA: improve top-k algorithm selection (#28713)
CUDA: radix top-k for large row counts Replaces CUB's per-row DeviceTopKKernel with a grid-over-rows radix select, gated on GGML_CUDA_TOPK_RADIX_MIN_ROWS. On qwen4exp at 34,816 tokens this cuts top-k from 1,671,253 launches / 5,761.8 ms to 2,329 / 941.8 ms. CUDA: select the…
31 -
b11509
CUDA: fix CCCL version guard breaking on major version rollover ( #29453 ) CUDA: fix CCCL version guard breaking on major version rollover The guard compared the major and minor components independently: CCCL_MAJOR_VERSION >= 3 && CCCL_MINOR_VERSION >= 1 Minor resets to 0…
24 -
b11507
llama: support MoE cache over multiple GPUs ( #30112 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53988921 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
29 -
b11505
vulkan : fix TOP_K for +inf/NaN inputs and k = 1 on negative values ( #30107 ) The bucket search in topk_nary_search.comp started from the range [0, 0xFF800000), which ends just below the ordered-uint mapping of +inf, so +inf and NaN were never counted. A workgroup block with…
16 -
b11510
CUDA : looped PAD kernel for more than 65535 rows or slices ( #30147 )
34 -
b11503
vendor : update cpp-httplib to 0.60.1 ( #30134 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53949333 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
29 -
b11501
sycl: FWHT optimizations ( #29605 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53938516 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
31