llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 22d ago
b11115
opencl: add bin kernel kernel_gemm_noshuffle_q4_k_q8_1_dp4a_ila_a8_bin ( #29056 ) opencl: add A8 Q4_K non-MoE dp4a binary kernel opencl: rename binary kernel selection helpers Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49361383…
11 -
b11114
server: fix router eviction races with the existing queue ( #29217 ) server: route every model load through the queue A model loaded by the fast path has no queue entry, so tick() evicts it at its LOADED transition before its own request is proxied. Every load now joins the…
20 -
b11113
server: do not pass log file to children ( #29212 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49344105 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
34 -
b11111
vulkan: add Intel Xe flash attention optimization kernels (2/3, Xe-LPG Plus/Xe2/Xe3) ( #24406 ) vulkan : Intel FA kernel optimization for split k path vulkan : Host code update for Intel split k FA kernel path selection, fix A770 Linux op test failures vulkan : use symmetric…
18 -
b11110
mtmd: add various sanity checks ( #29276 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49308522 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
10 -
b11109
metal : gate mul_mm_id src1 rescale behind ggml_prec ( #29029 ) metal : gate mul_mm_id src1 rescale behind ggml_prec Assisted-by: Claude Fable 5.1 ggml-webgpu: reject MUL_MAT_ID when src1 precision is F32 cuda/vulkan: reject MUL_MAT_ID in supports_op when src1 prec is F32 fix…
10 -
b11108
ggml : IQ1_M build prefix sums once per block ( #28706 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49294443 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
4 -
b11105
jinja: use const for statement::execute and ::visit ( #29271 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49279983 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
29 -
b11104
server: Add support for binding to multiple addresses ( #28690 ) Add support for binding llama-server to multiple addresses Assisted-by: Codex remove redundant thread handler make it clear about overlapping addr reject --port 0 with multiple tcp addr improve arg handler nits fix…
32 -
b11103
spec : support DFlash for HunyuanOCR ( #28890 ) model : add DFlash layer-input taps for HunyuanVL DFlash speculative decoding needs the target graph to expose the residual stream entering each layer (res->t_layer_inp[il]) - the draft model reads those tensors to build its…
4 -
b11102
convert: add MiMo-V2.6 support ( #29257 ) convert: add MiMo-V2.6 support Hoist the K3 mxfp4 conversion repack into base.py so it can be reused Remove decoder from mmproj convert Update conversion/mimo.py fix: use autoparser Co-authored-by: Sigbjørn Skjæret…
5 -
b11101
cmake : allow repeated find_package calls for llama ( #29228 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49213556 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
38 -
b11099
ci : publish snapdragon builds in release workflow ( #29007 ) The snapdragon CI builds packages only to feed the QDC device tests, so Hexagon NPU binaries never reached the releases page. Build both targets in release.yml and attach them as release assets. Website:…
15 -
b11097
opencl: add A8 Q4_0 non-MoE dp4a binary kernel ( #29055 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49141307 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
4 -
b11096
ci : update Level Zero SDK to v1.33.1 and enable the L0/oneDNN CMake …
11 -
b11095
hexagon: new HMX-optimized GATED_DELTA_NET ( #29199 ) hex-gdn: start putting together HMX support for GDN hex-gdn: working hmx but not-pipelined and slow for now hex-gdn: re-write vtcm layout handling and prep for pipelining hex-gdn: starting to pipeline hmx and dmas hex-gdn:…
11 -
b11094
vendor : update cpp-httplib to 0.57.1 ( #29239 ) Signed-off-by: Adrien Gallouët adrien@gallouet.fr Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49092442 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
17 -
b11093
metal : fix mask bounds in flash attention block pre-pass ( #29220 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/49087452 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
15 -
b11090
cuda: fix sm_70 tile compilation error ( #29224 ) The 5-argument load_ldmatrix added in 1884824 only defines tile<16,8>, so the Volta tile<8,4> does not match. See #29222 for details. Building on 1884824 , generalize the tile shape of the 5-argument load_ldmatrix from <16,8> to…
14 -
b11081
test-llama-archs : make tensor data stdev configurable and improve help ( #29133 ) test-llama-archs : make tensor data stdev configurable Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp test-llama-archs : expand usage and add examples Assisted-by:…
19 -
b11080
tests/test-backend-ops : allow regex entries in the -o filter ( #29204 ) tests/test-backend-ops : allow regex entries in the -o filter so far -o only accepted a comma separated list of exact op names or full test case strings. entries that are not plain op names are now treated…
31 -
b11078
args: add env vars for temperature, top-p, min-p and penalties ( #27380 ) Allow configuring --temp, --top-p, --min-p, --repeat-penalty, --presence-penalty and --frequency-penalty via LLAMA_ARG_* so llama-server can be fully controlled from an EnvironmentFile (e.g. systemd on…
18 -
b11077
server : do not forward --api-key-file to router-spawned child instances ( #28938 ) In router mode, authentication belongs to the router. unset_reserved_args() already unset LLAMA_API_KEY, but did not unset LLAMA_ARG_API_KEY_FILE. When --api-key-file was passed, children…
4 -
b11076
tests : remove stale comment ( #29140 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48998612 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
33 -
b11074
json: Fixed json enum handling ( #28518 ) Fixed json enum handling Added common_json_value handling for enum values. Added tests/test-json.cpp to cover testing of some aspects of common_json. Removed tests as requested. Applied recommended style and simplification Simplified by…
17 -
b11071
ci : Upgrade CUDA to 13.4 for Ubuntu CUDA Release Builds ( #29202 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48951751 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
35 -
b11070
hexagon: overhaul of buffer and DMA handling to support 64bit mappings + improvements ( #29197 ) hex-dma64: enable support extended buffer mappings and 64bit dma hex-dma64: expand binary ops to support more DMA scenarios hex-dma64: add binary-ops.h hex-dma64: add --hex-dma64 to…
8 -
b11069
cuda : tune MMVQ to MMQ crossover for SM70 (Volta) ( #28912 ) tune MMVQ to MMQ crossover for SM70 (Volta) Signed-off-by: Yangyu Chen cyy@cyyself.name Apply suggestion from @JohannesGaessler Apply suggestion from @JohannesGaessler Apply suggestion from @JohannesGaessler…
8 -
b11068
metal : fix deprecation warnings from macOS 27 SDK ( #29136 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48890258 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
29 -
b11067
webgpu : add fused gdn + cpy ( #28976 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48873164 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
20 -
b11065
CUDA: tune FA for Gemma 4 on Ampere or newer ( #29152 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48802880 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
28 -
b11064
metal : support arbitrary hc in dsv4_hc_pre ( #29169 ) the dsv4_hc_pre kernels hardcoded hc = 4 via a constexpr used with simd_shuffle, so the op was rejected by supports_op for any other hc and fell back to CPU. Kimi-K3 uses dsv4_hc_pre with hc equal to the number of banked…
17 -
b11062
CUDA: enable sparse fa for qwen4 ( #28770 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48739244 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
35 -
b11059
metal: add F16 input to the FWHT ( #29094 ) metal: add F16 input to the FWHT The Metal FWHT kernel accepts F32 input only. This change makes the source type a template parameter, so the kernel reads an F16 source directly instead of requiring a converted copy. The F32…
11 -
b11057
chat : add dedicated Ling 3.0 (Bailing V3) parser ( #28682 ) chat: add dedicated Ling 3.0 (Bailing V3) parser Ling 3.0 Flash templates pre-open the think block in the generation prompt, so the model never emits an opening , and a tool call can arrive before any . The generated…
4 -
b11056
hexagon: enable I32 GET_ROWS ( #29116 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48662388 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
29 -
b11055
hexagon: add support for GEGLU_QUICK ( #29114 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48660160 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
14 -
b11054
hexagon: enable support for TOP_K op ( #29113 ) hexagon: enable support for TOP_K op hex-topk: thread single-row TOP_K, raise VTCM-based size cap hex-topk: fix TOP_K mdev row partitioning hex-topk: optimize TOP_K large-row selection hexagon: clean up comment formatting hex-docs:…
24 -
b11053
server : improve startup log messages ( #29125 ) server-models : show source per model in log Show [source] tag (preset/models_dir/cache) per model instead of cryptic * marker Show HF hub cache path in the 'Loaded cached model presets' log Add hf_cache::get_cache_dir() public…
14 -
b11052
json-schema : accept escaped hyphen in regex patterns ( #29127 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48635641 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
14 -
b11050
metal : fix FA support checks ( #29122 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48626162 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
26 -
b11049
test-llama-archs : generate dummy test vocab ( #29084 ) Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/48623939 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
37 -
b11048
metal : support qwen4exp hc ops ( #29000 ) Add support for the new DSV4 HC op variants used by qwen4exp: hc_pre with per-element sigmoid gate (gated variant) hc_post with identity mixing (comb == nullptr) Assisted-by: pi:llama.cpp/Qwen3.8-27B Website: https://llama.app…
18