llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 1mo ago
b10202
sycl: fuse RMS_NORM + MUL ( #26015 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan)…
29 -
b10201
ggml-webgpu: improve flash_attn_vec for quantized KV at long contexts ( #25956 ) improve fa of quantized kv cache Fix some bugs and some comments. fix v type check and some comments Fix build error caused by rebasing editorconfig checking pass Website: https://llama.app…
13 -
b10199
server: support inp embd to generate next token ( #26313 ) server: support embd for sampled token fix ~server_batch() Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
10 -
b10200
mtmd: add lanczos resize method [no release] ( #26341 )
21 -
b10198
vulkan: Support quantized concat ( #25684 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
21 -
b10197
Test support for alternative conv layout ( #25617 ) add bool cwhn = true to conv_2d test cases add layout check at graph building time extend layout checks for conv2d.cu kernel in CPU back-end kernel needs to be stored contiguously to prevent test failures with cwhn=1 trim white…
6 -
b10196
llama-context : sync pending async copies before clearing embd_seq ( #25676 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
20 -
b10195
tests : avoid building get-model.cpp many times ( #26317 ) tests : remove get-model.cpp tests : fix quant type selection Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
13 -
b10194
ggml-cuda: Allow transpose-free gemmv computation ( #26171 ) When matrix's weights are shaped 1xK is leverage a transpose-free computation to use mat_mul_vec_f. Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
20 -
b10192
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
27 -
b10189
Remove custom cpu op from the M3 graph, express with stock ops ( #26297 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu…
23 -
b10188
metal: fix memory unwire if model is freed without any GPU operations ( #26082 ) metal: fix memory leak if model is freed without any GPU operations metal: run dummy work only if residency sets are used metal: wrap function in #if defined metal: measure system-wide wired memory…
6 -
b10186
ggml : Fix issue with kleidiai ci and stringop overflow warning ( #26277 ) Signed-off-by: Jonathan Clohessy Jonathan.Clohessy@arm.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
28 -
b10184
mimo2: address MTP review feedback ( #26228 ) Co-authored-by: tnhnyc 115956684+tnhnyc@users.noreply.github.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
38 -
b10182
llama: move suppress_tokens handling to common/sampling ( #26276 ) llama: move suppress_tokens handling to common/sampling address security issues rm has_logit_bias Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
23 -
b10181
ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory ( #26141 ) ggml_cuda_should_use_mmq() selects MMQ purely from the quantization type. The current MMQ configurations are designed and maintained against a minimum of 48 KiB per-block shared memory, the limit…
36 -
b10180
sycl: contiguous fast path + 32-bit index math for unary elementwise ops ( #25946 ) sycl: contiguous fast path + 32-bit index math for unary elementwise ops sycl: use fastdiv for elementwise index math Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
8 -
b10179
vendor: update BoringSSL to 0.20260728.0 ( #26241 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
26 -
b10178
server : add trace logging for slot similarity checking ( #26271 ) Adds trace logging in server-context.cpp for slot similarity checking during prompt cache slot selection, including skip reasons and similarity calculation details. Assisted-by: llama.cpp:Qwen3.6-27B Website:…
21 -
b10176
RPC: add tensor_memset ( #25912 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu…
37 -
b10175
add rdna3.5, and 3 to mmq configs so they can be tuned independently. ( #26199 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
26 -
b10174
model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) ( #25980 ) model: add NextN/MTP speculative decoding support for GLM_DSA (GLM-5.2) Adds GLM-5.2 NextN/MTP as a --spec-type draft-mtp target: nextn tensor loading via the qwen35moe/step35-style presence…
29 -
b10173
model: Add Laguna-S-2.1 LLM_TYPE ( #26233 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
5 -
b10172
ggml-webgpu: Fix some binding alias issues to support all archs, fix recurrent-state-rollback test ( #25931 ) Add overlap glu variant to support all archs, fix recurrent-state-rollback test format Fix all arch overlapped ranges format diagnose bus error on apple ci More testing…
25 -
b10171
opencl: skip the Adreno KQ/KQV image kernels for multi-stream batches ( #26189 ) The Adreno KQ/KQV image1d kernels (ggml_cl_mul_mat_kq_kqv_adreno) ignore dim 3 entirely: the sub-buffer covers only nb02*ne02 bytes and the kernel receives no ne03/ne13/nb03/nb13 arguments. With the…
19 -
b10167
server: abstract llama_memory calls to common_memory ( #26221 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
26 -
b10166
ggml : set output of view src ( #25729 ) llama-graph: set_outputs to t->view_src change set_output to GGML_ASSERT about views not being outputs sampler : avoid views in outputs cont : fix dist sampler cont : consistent logits handling ggml : set output of view src graph :…
28 -
b10165
vulkan: add iq4_nl support back to FA ( #24585 ) vulkan: add iq4_nl support back to FA I was originally concerned about wasting shared memory on the LUT, but it's small and unlikely to matter in practice. Also support q1_0 for non-coopmat2. Fixes #23681 remove q1_0 FA support…
36 -
b10164
ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration ( #22675 ) ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration cuda: added SSD CICD fixes for CUDA / HIP / MUSA / MSVC. ggml-cuda: review comments fixed. ggml-cuda: Fuse M matrix materialization…
23 -
b10158
spec: add eagle3-v3 support for gpt-oss model ( #25794 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
23 -
b10156
Disable -ffast-math on HIP ( #25495 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan)…
5 -
b10154
common : add common_print_available_devices() ( #26170 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
13 -
b10153
model: Add support for Nanbeige4.2 ( #25994 ) support nanbeige4.2 model fix fix flake8 Lint check fix loop bound check and drop redundant head_dim Co-authored-by: root lizongqiang@kanzhun.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
18 -
b10152
fit : count nextn (MTP) blocks in n_gpu_layers so front layers stay on GPU ( #26177 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
8 -
b10151
sycl(build): parallelize ocloc invocations ( #25903 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
5 -
b10150
ggml : adjust logic for offloading ops to weight's backend ( #25832 ) ggml : adjust logic for offloading ops to weight's backend llama : dsv4 graph fixes Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
36 -
b10149
tests : remove unnecessary sync in test-save-load-state ( #26166 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
34 -
b10148
common: fix explicit -md precedence over draft sidecar resolution ( #26165 ) common: fix explicit -md precedence over draft sidecar resolution Follow-up of #25955 , an explicit --model-draft file given with -hfd was silently overridden by the sidecar resolution of the draft…
28 -
b10146
ggml-cpu: Enable BF16 tiled gemm optimization on PowerPC ( #26068 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
9 -
b10145
args: add -lm mlock where it mlocks but doesnt mmap ( #26135 ) arg: add -lm mlock where it mlocks but doesnt mmap Signed-off-by: Aaron Teo aaron.teo1@ibm.com docs: rm unwanted docs changes Signed-off-by: Aaron Teo aaron.teo1@ibm.com docs: revert auto-formatting Signed-off-by:…
35 -
b10144
server + ui: fix stream routes for model names containing a slash ( #26137 ) server + ui: refactor resumable stream routes to query string conv_id The conversation id can embed a model name containing slashes (ggml-org/...) in router mode, which the decoded path splits before…
37 -
b10142
mtmd: Add Vision Support for Minimax-M3 ( #25113 ) Add preliminary MiniMax-M3 support Text-only port that re-uses existing components: MiniMax-M2 style GQA with per-head QK-norm and partial rotary, DeepSeek-V3 style leading-dense and routed/shared experts, and swigluoai…
18 -
b10141
mtmd: fix android build ( #26150 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu…
21 -
b10133
ui: fix context gauge card regressions and land at the conversation e…
5 -
b10121: ui: reduce per-token render cost when streaming (#26053)
performance harness - the empirical root Assisted-by: Claude Opus 4.8 210.36ms -> 2.67ms per streamed token Assisted-by: Claude Opus 4.8 11.58ms -> 0.62ms per streamed token Assisted-by: Claude Opus 4.8 22.02ms -> 3.33ms per streamed token Assisted-by: Claude Opus 4.8 3.07ms ->…
6 -
b10107
hexagon: fix Windows crash when op_poll is enabled ( #26029 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
8 -
b10106
CUDA: fix external compilation of q1_0 MMQ ( #25778 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
4 -
b10105
args: refactor mlock/mmap/directio into load-mode ( #20834 ) args: overhaul mmap/mlock/dio into single arg Signed-off-by: Aaron Teo aaron.teo1@ibm.com docs: update docs with llama-gen-docs Signed-off-by: Aaron Teo aaron.teo1@ibm.com chore: satisfy code quality Signed-off-by:…
22