llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 2h ago
b10690
memory : copy Hadamard matrix to k_rot tensor only if it has buffer assigned to prevent crashes during context shift of unquantized K cache ( #27967 ) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Co-authored-by: AesSedai 7980540+AesSedai@users.noreply.github.com…
5 -
b10689
ggml: allow passing alloc dependencies in graph_optimize ( #27301 ) ggml: allow passing alloc dependencies in graph_optimize add alloc dep tests add TODO about using flat array Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43919110…
23 -
b10688
metal : add fa-vec tunings for M2 ( #27940 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43902029 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
12 -
b10687
opencl: use a better matmul path on two Adreno GPU generations ( #27640 ) opencl: default the Adreno xmem F16xF32 GEMM on for X2E kernel_mul_mm_f16_f32_l4_lm is the slowest matmul this backend has on Adreno: on the X2-90 it runs the gpt-oss-20b attention projections at roughly a…
27 -
b10686
metal : assert shared memory padding ( #27951 ) metal : assert shared memory padding cont : add ref Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43867822 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
28 -
b10685
metal : add remaining fa-vec tunings for M4 Pro ( #27915 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43865923 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
4 -
b10684
sycl: make --fit respect --fit-target better ( #27629 ) improve the --fit algorithm to take into account the actual peak required VRAM for a given context size on a SYCL backend. This includes both properly accounting for how much VRAM is required when the allocated context is…
20 -
b10682
metal : add fa-vec tunings for M1 Max ( #27932 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43859747 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
24 -
b10681
vulkan: Change mul_mat_id to pad K rather than N ( #27925 ) The N padding is needed for mul_mat, but not mul_mat_id. For mul_mat_id, we indirect the row index through a shared memory lookup table which avoids any OOB row coordinate. But that callback doesn't bounds check K, so…
24 -
b10680: snapdragon: python SDK setup (Windows) (#27903)
port setup-build.ps1 to setup_sdk.py, to facilitate installation of Hexagon and OpenCL SDKs on Windows rename setup_sdk.py -> setup-sdk.py flake8 fix: print() -> logger.info() Co-authored-by: Kristopher Urquhart kurquhar@qti.qualcom.com
33 -
b10679
bench: add --tensor-read-lazy ( #27881 ) bench: add --tensor-read-lazy rm the alias rename to LLAMA_LAZY_MODE_* Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43739963 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
17 -
b10678
model: qwen4exp: reduce number of graph splits ( #27880 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43734155 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
23 -
b10677
vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize ( #27812 ) vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize is_src_of doesn't treat two views of one tensor as dependent, so the optimizer reorders nodes across aliased reads and writes.…
12 -
b10676
ggml : fix conv_transpose_2d for multiple batches ( #26132 ) ggml : fix conv_transpose_2d for multiple batches ggml_compute_forward_conv_transpose_2d_impl only computed the first batch (ne[3] of the destination); every batch after the first was left as zero. Both the src1…
10 -
b10675
Vulkan: add hoisting support for row IDs and expert count in shaders ( #26686 ) vulkan: add hoisting support for row IDs and expert count in shaders use hoisted row ids in coopmat2 vulkan: address review feedback on count_experts use vk_op_count_experts_push_constants instead of…
23 -
b10673
metal : add fa-vec tunings for M4 ( #27875 ) This adds fa_vec_tuned_table records for Apple M4 to ggml-metal-tuning.cpp. Includes F16, Q4_0, Q4_1, Q5_0, Q5_1, and Q8_0. (M4, 10 GPU Cores) Co-authored-by: Strongtut 8432058+Strongtut@users.noreply.github.com Website:…
13 -
b10672
OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU, and new ops ( #27843 ) OpenVINO Backend: Fuse IM2COL + MatMul convolution into OpenVINO convolution ci:ggml-ov: Skip recurrent state rollback tests ci:ggml-ov: Skip recurrent state rollback tests Update…
33 -
b10670
sycl: use TILE for quantized KV decode on BMG ( #26689 ) Route quantized KV decode to TILE on Xe2 (BMG) only, keep VEC on other archs until validated there. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43660247 macOS/iOS: macOS…
10 -
b10669
sycl: bind the f16 KV cache in place for the oneDNN SDPA path ( #27468 ) Measured at a live KV length of 34816 (32768 depth plus one 2048 ubatch), on Qwen3.8 27B Q4_K_S: per tensor 4 * 34816 * 256 * 2 B = 71.3 MB staged per call K and V, so 2x = 142.6 MB traffic per call read…
9 -
b10668
metal : add fa-vec tunings for M3 Max, M5 and M5 Pro ( #27863 ) metal : add fa-vec tunings for M5 This is a followup contribution to efeda76 as requested in https://github.com/ggml-org/llama.cpp/discussions/27668 to add support for additional Apple GPUs. I generated this output…
22 -
b10667
metal : add fa-vec tunings for M4 Pro ( #27824 ) This is a followup contribution to efeda76 as requested in https://github.com/ggml-org/llama.cpp/discussions/27668 to add support for additional Apple GPUs. I generated this output using the provided instructions: git clone…
36 -
b10666: tests : run test-save-load-state across all architectures (#27755)
tests : run test-save-load-state across all architectures test-save-load-state previously only ran in ctest against a single downloaded model (tinyllamas/stories15M), i.e. only the llama arch. Add a --models DIR mode to test-save-load-state that runs the full save/load suite…
20 -
b10665
model: add DSpark support for Nemotron3.5 ( #27804 ) model: add DSpark support for Nemotron3.5 Update src/models/dflash.cpp Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Co-authored-by: Xuan…
13 -
b10663
hex-unary: fix RMS_NORM_MUL weight-offset bugs for grouped/broadcast norms ( #27798 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43526462 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
17 -
b10662
server: add ctx-per-slot (--kv-unified-per-slot) ( #24124 ) Add ctx-per-slot argument for unifid KV cache Swap out ctx fractions for ctx pool slots Formatting cleanup Remove ctx-pool-slots, make ctx-per-slot an int refactor it Co-authored-by: Xuan Son Nguyen son@huggingface.co…
19 -
b10661
ci : build only the ggml-hip backend for windows-rocm release ( #27753 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43506547 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
15 -
b10660
model: add Qwen3.8-Flash-Next (qwen4exp) ( #27742 ) gguf: add qwen4exp (Qwen3.8-Flash-Next) arch and converter Adds the GGUF-side plumbing for HF model_type qwen4_exp: MODEL_ARCH.QWEN4EXP plus tensors for the low-rank hyper-connection variant (hc_ norm/down/up/inject) and the…
29 -
b10659
ci : bundle HIP runtime DLLs with Windows ROCm release ( #26973 ) Copy amdhip64_7, amd_comgr and rocm_kpack next to the binaries so the correct HIP runtime loads over the driver's copy in System32. Fixes #26929 . Website: https://llama.app Attestations:…
34 -
b10658
spec : add DFlash2 support (local convolution + candidate selector) ( #27342 ) ( #27816 ) spec : add DFlash2 support (local convolution + candidate selector) ( #27342 ) support DFlash2 Add p_min in DFlash2 Assisted-by: Claude Opus 5 Revert unnecessary changes Assisted-by: Claude…
35 -
b10657
opencl: add bin kernels kernel_gemm_moe_q4_0_q8_1_dp4a_bin , kernel_gemm_moe_mxfp4_q8_1_dp4a_bin ( #27768 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43474540 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
13 -
b10656
quantize: cap working memory size to avoid loading big tensors onto RAM ( #27795 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43468853 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
35 -
b10655
Feature: Added LIGHTNING_INDEXER support for Deepseek V4 ops on Vulkan Backend ( #27453 ) vulkan: add LIGHTNING_INDEXER op vulkan: updated lightning_indexer.comp and ggml-vulkan.cpp with 128-lane dot-product reduction moved from a shared-memory tree to subgroupAdd. vulkan:…
14 -
b10653
llama: model_loader: add TENSOR_READ_LAZY ( #27794 ) llama: model_loader: add TENSOR_GET_ROW_LAZY add --tensor-read-lazy rename to TENSOR_READ_LAZY gen docs address comments Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43411286…
32 -
b10649
spec: Add benchmark-only synthetic speculative acceptance options ( #27711 ) Add benchmark-only synthetic speculative acceptance to llama-server and llama-cli Address review comments Address review comments Add some comments in the code Website: https://llama.app Attestations:…
34 -
b10648
model : simplify MiniMax-01 graph ( #27790 ) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43387702 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
15 -
b10647
args: add --video-* CLI arguments ( #24318 ) args: add --video-* CLI arguments gen docs nits add mtmd_helper_init_opt Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43380700 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
33 -
b10646
metal : fix memory leaks due to missing autoreleasepools ( #27758 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43372015 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
20 -
b10645
llama : add --n-cpu-ffn option ( #26622 ) common : dedupe --n-cpu-moe / --spec-draft-n-cpu-moe override loops common : add --n-cpu-ffn to CPU-offload dense FFN weights of first N layers common : generalize llm_ffn_block_regex over the FFN regex, drop TODO Website:…
34 -
b10644
models : support nanbeige4.2-3B ( #27730 ) Co-authored-by: admin lizongqiang@kanzhun.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43308396 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
19 -
b10643
hexagon: support for multi-NPU devices (IQ9, IQ10) and fully asynchronous backend ( #26501 ) hexagon: use non-host bufs by default and make the backend fully async hex-hb: remove optional hostbuf support and fix async copy hex-unary: relax supported unary check hex-bufs: use…
34 -
b10642
llama: add token ID tracking to KV cell ( #27762 ) kv: track token id rm get_prev_tokens, move it to the main pr nits add get_prev_tokens Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43245878 macOS/iOS: macOS Apple Silicon (arm64)…
15 -
b10639
vulkan: warptiles currently assume warp sizes <= 64, clamp to work around larger warps ( #27726 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43173045 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
5 -
b10638
Implemented vulkan cross_entropy_loss and cross_entropy_loss_back ( #27216 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43166320 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
33 -
b10636
ci: Clean up UI builds from releases ( #27706 ) ci : inline UI version resolution into ui-build.yml ci : build UI once and reuse the artifact in release jobs Server jobs now extract the ui-build artifact into tools/ui/dist instead of npm-building the UI. Also removes the…
38 -
b10632
ggml-metal: add chunked SSD MMA for Mamba-2 prefill optimization ( #26647 ) metal: WIP chunked SSD SSM_SCAN kernels for multi-token prefill metal: drop scalar SSD path; MMA + sequential tail drop WIP ssm scan test noise remove state_from_dst and rename CS and NSG constants…
13 -
b10631
ggml-meta: propagate buffer usage and call init on the new tensors ( #27586 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/43044002 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
29 -
b10630
kleidiai: Rework KleidiAI Build System/Integration ( #26077 ) Rework KleidiAI Build System/Integration Signed-off-by: Jonathan Clohessy Jonathan.Clohessy@arm.com Add fp16 guard, and fix cmake caching issue Signed-off-by: Jonathan Clohessy Jonathan.Clohessy@arm.com Fix…
32