llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 15d ago
b10431
ggml : recurrent state rollback for ggml_ssm_scan ( #26623 ) Initial changes for Recurrent state rollback for nemotron for cpu and cuda Removing CPU RS rollback. Will enable it in subsequent PRs addition of test case Removing assert and calling runtime API to check if op is…
21 -
b10430
llama : allow virtual igpu devices ( #26953 ) llama : allow virtual igpu devices cont : better comment Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
26 -
b10429
server: allow accessing /metrics and /slots during llama_decode() ( #27041 ) server_queue::worker call llama_decode inside yield_to_queue also handle process_mtmd_chunk clean up nits rm test Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
13 -
b10428
tests : replace personal home directory paths with generic placeholders ( #27043 ) Scrub developer-specific /home// paths from example docs and test fixtures so they don't leak into the tree. examples/test-cmake/README.md: /home/danbev/... -> /path/to/llama.cpp/...…
11 -
b10427
sycl: fuse mul_mat(gate) + mul_mat(up) + GLU for q4_K dense FFN ( #26779 ) Measured on Arc Pro B70 (Battlemage, Level Zero), llama-bench -r 20, two interleaved rounds, tg128: qwen2.5-3B-Instruct Q4_K_M 154.18 -> 158.53 t/s +2.8% gemma-2-2b-it Q4_K_M 162.45 -> 165.62 t/s +2.0%…
23 -
b10426
ggml: force single thread on wasi ( #25686 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
27 -
b10425
sycl: fuse the gated-delta-net state writeback cpy ( #26643 ) Port of #23940 . Arc Pro B70, Qwen 3.6 27B Q4_K - Medium (48 of its 64 blocks run gated_delta_net), -ngl 99 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 2048, interleaved A/B passes of r=3: tg128 23.91 / 23.90 / 23.90 -> 24.19…
26 -
b10424
dflash : clarify output logging of target_layer_ids ( #27013 ) This commit tries to make the logging of target_layer_ids a bit clearer and easier to read. Currently the output generated looks like this: 0.00.468.624 D load_arch_hparams: DFlash extract_layers = [0.00.468.626 D 2,…
25 -
b10423
common: apply CPU parameters across tools ( #27026 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
9 -
b10419
OpenVINO: Qwen3.5, memory optimization, and test-recurrent-state-rollback ( #26952 ) OpenVINO backend: 1) enable gpt-oss moe on OV bk; 2) enable mxfp4 support OpenVINO backend: disable TOPK_MOE op test OpenVINO Backend: Add op FILL support OpenVINO backend: enable set rows with…
36 -
b10418
[SYCL] Support host pinned mem to improve SYCL Host-to-Device Memory Access ( #26789 ) support host pinned mem, ggml_backend_sycl_host_buffer_type_get_max_size, fix the thread-safe issue Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
6 -
b10417
chat : fix LFM2 tool call arg name prefix ambiguity ( #26960 ) Assisted-by: Claude Opus 5 Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu…
6 -
b10416
server : serve index.html with no-cache ( #27006 ) index.html was served with max-age=31536000, immutable like the hashed assets, but its name is stable while its contents change every build, so a cached copy pins the UI to an old build. It now revalidates via its existing ETag,…
7 -
b10415
spec : auto-detect mtp draft model type ( #27005 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
37 -
b10414
metal : add TQ2_0 support ( #26980 ) metal: add TQ2_0 support Add support for the GGML_TYPE_TQ2_0 (ternary, 2 bits per element) type in the Metal backend. Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731 cont : optimize mul_mv kernel float ops over integer ops precalculate sums…
29 -
b10413
common : auto-detect spec type from draft GGUF metadata ( #26814 ) common : auto-detect spec type from draft GGUF metadata When -md loads a local draft model without --spec-type, the sidecar inference in common_models_handler_apply only checks HF repo sidecars and misses local…
31 -
b10412
spec: enable backend sampling for both dflash & dspark ( #26958 ) dflash: enable backend sampling for both dflash & dspark enable p_min > 0 in backend sampling and add guard cont : add TODO Co-authored-by: Georgi Gerganov ggerganov@gmail.com Website: https://llama.app macOS/iOS:…
21 -
b10411
ggml-cpu/ops: vectorize flash-attention V-cache F16 to F32 conversion ( #26947 ) Co-authored-by: jinzihao jinzihao.jzh@alibaba-inc.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
9 -
b10410
sycl: remove separate fp32 type promotion in gemm non-oneDNN path ( #26372 ) sycl: use automatic fp16 promotion in gemm sycl: remove redundant comment Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
22 -
b10408
sycl : Add DMMV ESIMD Q3_K kernel ( #26251 ) Add DMMV Q4_K and Q6_K ESIMD kernels Configure cmake build with -DGGML_SYCL_ESIMD=ON to enable. Signed-off-by: Todd Malsbary todd.malsbary@intel.com Refactor ESIMD kernels to share common code Signed-off-by: Todd Malsbary…
16 -
b10405
ggml-hip : remove -funsafe-math-optimizations ( #26696 ) It enables -fassociative-math, which reassociates FP reductions and can flip greedy argmax on RDNA3.5 (e.g. MTP speculative decode diverging from the non-speculative baseline). Drop it so HIP builds are IEEE-conformant.…
11 -
b10400
ggml : fix arm builds, unused var ( #26991 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
18 -
b10398: common: add system-level config file (#26118)
common: Add CLI > ENV > models-presets > INI precedence CLI flags have the highest precedence ENV vars have the second-highest precedence System and User configs have the lowest precedence Linux/BSD/Mac /etc/llama.cpp/config.ini <…
21 -
b10375
chat : tighten bare function parsing for Qwen models ( #26793 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
28 -
b10373
imatrix.cpp: Move finite check and only check touched experts ( #26861 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu…
5 -
b10369
mtmd: support pocket-tts ( #26871 ) adapt the api text model ok working impl, need verify and clean up mtmd: build the pocket-tts transposed convolutions as GEMM + col2im ggml_conv_transpose_1d has no grouped mode, so the depthwise upsample was built as one convolution and one…
16 -
b10380: chat : fix muse-glimmer detection of tool calls after EOM (#26879)
chat : fix muse-glimmer swallowing a trailing tool call into content Muse Glimmer routinely answers the user and calls a tool in a single generation. The template terminates a message with <|eom|> when more messages follow in the same turn and <|eot|> only at the end of the…
29 -
b10361
model : fix SWA not being enabled for EXAONE 4.5 ( #26848 ) model : fix SWA not being enabled for EXAONE 4.5 load_arch_hparams tests hparams.n_layer() == 64 before LLM_KV_NEXTN_PREDICT_LAYERS has been read. n_layer() returns n_layer_all - n_layer_nextn and n_layer_nextn defaults…
16 -
b10360
common/peg : suppress incomplete escape sequences ( #26780 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
25 -
b10359
ggml-webgpu: fix CI errors from #25025 and #25262 ( #26566 ) test new flash_attn test rebase and fix to disable subgrou matrices when max_kv_tile == 0 delete log output Add i32 support to cpy and enables the all ops test restore the non target ci tests comment out of TODO of…
26 -
b10358
Address review comment of PR 25532 ( #26852 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
35 -
b10357
opencl: transpose the K tile in local memory for FA prefill kernels ( #26428 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
20 -
b10362: tests : disable backend sampler hip multi output (#26878)
test-backend-sampler: skip multi_output_sampling_chain on HIP The new multi_output_sampling_chain test uses top_k, whose backend probs path needs CUB (unavailable on HIP), so sampled_probs is null and the test aborts. Add it to the existing HIP skip list alongside the other…
29 -
b10356
ci : target ROCm 7.14 for build and release ( #25775 ) Switch ROCm from 7.2.1 to 7.14 ROCm 7.14 is the first production release using TheRock build system. It can be installed using multi-arch deliverables from wheels, debs, rpms, tarballs or runfiles. Adjust ROCm targets for…
18 -
b10355
llama : support multi-output backend sampling ( #25532 ) Enable backend sampling with token speculation Clamp the mask sum before converting it into the sampled index Add a numeric context parameter declaring the maximum outputs one sequence More fixes Don't reuse memory for…
8 -
b10354
ggml-cpu : fix CPU affinity mask being ignored on Android ( #26838 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
37 -
b10353
ggml : require contiguous src for ROLL on CUDA and Metal ( #25928 ) ggml_roll only asserts nb[0] == ggml_type_size, so a permuted src is a valid input, but the CUDA and Metal roll kernels index by ne alone and never read the nb strides. A non-contiguous src therefore produced…
25 -
b10344
model: add MTP support for Nemotron model ( #26725 ) model: add MTP support for Nemotron Nano model model: add mtp_flags for nemotron model address review comments Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
23 -
b10343
vendor : update cpp-httplib to 0.53.0 ( #26821 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
5 -
b10342
model : Granite-Switch Architecture ( #25107 ) granite-switch: add llama.cpp backend (POC, CPU) New "granite-switch" architecture: a dense, all-attention Granite-4.1 model with N embedded LoRA adapters selected per-token by control tokens. gguf-py schema (arch, KV keys, stacked…
22 -
b10338
model-saver : fix expert shared/chunk FFN length key clobber ( #26693 ) The saver called add_kv with LLM_KV_EXPERT_SHARED_FEED_FORWARD_LENGTH twice, the second time passing n_ff_chexp. gguf_set_val_u32 removes-then-appends, so the second call clobbers the first: the saved…
5 -
b10336
ggml-webgpu : refactor several wgsl files and simplify flash_attn wgsl. ( #26134 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
15 -
b10335
ui: degrade the working directory picker when file search is off ( #26 …
18 -
b10333
ggml-cpu : fix missing Q5_0 dispatch in SpaceMiT backend ( #26792 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
4 -
b10332
ci: rm GGML_HIP_ROCWMMA_FATTN ( #26760 ) Signed-off-by: Aaron Teo aaron.teo1@ibm.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64…
10 -
b10331
server: report the isolate working directory from get_info ( #26773 ) server: report the isolate working directory from get_info Without an explicit cwd, get_info fell back to the server process working directory even when a tools runtime was configured. That named a host path…
19 -
b10330
CUDA: fuse rms_norm + mul + rope (+ view + set_rows) ( #26767 ) CUDA: fuse rms_norm + mul + rope (+ view + set_rows) tests: add broadcast weight case to rms_norm_mul_rope CUDA: check memory ranges before rms_norm rope fusion CUDA: check memory ranges in rope set_rows fusion…
25 -
b10329
server, ui: only offer a working directory when a tool reads it ( #26762 ) The working directory chip showed up as soon as the server exposed any builtin tool, so a server started with just get_datetime, or a user who turned every filesystem tool off in the settings, still got a…
8 -
b10328
server: add initial tool isolation support (via docker) ( #26507 ) server: add initial tool isolation support (via docker) add docs adapt get_info py: fix type check cont separate tools_io_sandbox / tools_io_docker rename sandbox --> isolate x-tool-docker --> x-tool-runtime…
15 -
b10327
CUDA: fix thread/block count in quantized cpy kernel launches ( #26731 ) CUDA: fix thread/block count in quantized cpy kernel launches tests: add uneven block count cpy case Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
31