llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 1mo ago
b10108
hexagon: further improved pipeline of the core bits (L2, DMA, MM, FA)…
38 -
b10099
CUDA: Improve NVFP4 W4A4 activation quantization ( #25730 ) Squash history before conflict-resolution during rebase on master WIP commit Add 32-byte loads, restore per-block amax Use nvfp4x4 intrinsic when available Fuse per-channel amax and quantization kernels Do pointer…
6 -
b10098
hexagon: activation ops update ( #25974 ) hex-geglu: optimized all-in-one geglu microkernel hex-geglu: enable non-contiguous src and strided DMA hex-act: enable non-contiguous srs and strided DMA for rest of ACT ops hex-act: generalize GLU per-thread functions via…
28 -
b10094
common: infer the speculative type from the draft repo sidecars ( #25989 ) With -hfd pointing to a repo that ships mtp-/dflash-/eagle3- sidecars and no --spec-type given, the draft resolved to a full model while the sidecar was the intended draft. When the speculative types are…
19 -
b10093
Fix DeepSeek4 crafted template ( #25414 ) chat: fix DS4 template to explicitly follow reference behavior Support DeepSeekv4 flag ( drop_reasoning ). fix: hook DS3.2 parser for DS4 as well fix: add tool result reordering fix: post-merge Website: https://llama.app macOS/iOS: macOS…
20 -
b10092
ggml: enable PowerPC backend variants on AIX ( #25983 ) ggml: enable PowerPC backend variants on AIX Allow the PowerPC CPU backend variants to be built on AIX by extending the platform check in the CMake configuration. This reuses the existing PowerPC backend implementations…
5 -
b10103
metal : add f16 type support to leaky relu ( #25981 )
23 -
b10091
ci : fix SYCL package shared library lookup ( #25987 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
5 -
b10090
webgpu : add CONV_2D_DW (depthwise conv2d) kernel ( #25847 ) webgpu : add CONV_2D_DW (depthwise conv2d) kernel Implement GGML_OP_CONV_2D_DW for the WebGPU backend, ported from the Vulkan backend's conv2d_dw.comp. Assisted-by: Claude Opus-4.8 Remove unnecessary comments in webgpu…
37 -
b10089
cuda: GET_ROWS quants ( #25962 ) cuda: add k-quant support to GET_ROWS Device-side embedding lookups require GET_ROWS to handle the k-quants used by common GGUF recipes (Q4_K_M stores token_embd as q6_K). Without it the backend rejects the op and the scheduler falls back to the…
31 -
b10087
Add support for Laguna XS.2 & M.1 ( #25165 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
7 -
b10085
mtmd : use align_corners for qwen3vl vision position embedding interpolation ( #25781 ) The Qwen3-VL learned position embedding is interpolated to the runtime patch grid with the default bilinear+antialias (align_corners=False) sampling, while the transformers reference uses…
32 -
b10084
hexagon: check tensor type when reusing descriptors ( #25968 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU)…
38 -
b10083
cuda: add sqrt_softplus in topk-moe for dsv4 ( #25896 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu…
33 -
b10082
kleidiai : warn once when a weight type has no KleidiAI kernel ( #25701 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu…
21 -
b10081
common: resolve draft repo to its requested sidecar ( #25955 ) With -hfd pointing to a repo shipping speculative sidecars, the draft resolved to the main model of that repo, since find_best_model() excludes sidecar files, and the explicit draft plan suppressed the sidecar…
26 -
b10080
server: return 400 instead of 500 on validation error with X-Conversation-Id ( #25760 ) server: return 400 instead of 500 on validation error with X-Conversation-Id set_req() attaches the spipe as soon as the header is present, before the request body is parsed. When params…
5 -
b10079
server : properly handle null llama_context ( #25868 ) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu…
11 -
b10078
vulkan: Refactor vk_queue to use per-instance mutexes and unique handles ( #23570 ) Refactor vk_queue to use per-instance mutexes and unique handles integrates VK_KHR_internally_synchronized_queues, abstracting the queue submission into a polymorphic interface that completely…
31 -
b10077
ggml-openvino: Add GGML_BACKEND_DL_IMPL invocation for OpenVINO backend ( #25795 ) This adds the missing GGML_BACKEND_DL_IMPL() macro invocation, that other backends have. Fixes #25586 for me Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
6 -
b10076
CUDA: vectorize same-type get_rows with int4 copy ( #25929 ) k_get_rows_float did a scalar one-element-per-thread copy and recomputed the row-invariant work (index load, fast_div_modulo, src/dst row pointers) for every element. Hoist that out of the per-element loop, and add a…
26 -
b10075
hexagon: add CLAMP op ( #25934 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu…
19 -
b10069
opencl: Support broadcast for Adreno MUL_MAT and honor view_offs for Adreno Q8_0 MUL_MAT for llama-server multi-stream ( #25910 ) opencl: handle broadcast for adreno gemm/gemv_noshuffle opencl: honor view_offs for adreno noshuffle gemm/gemv opencl: general GEMM/GEMV support…
14 -
b10068
model: rotate injected K/V cache for DFlash ( #25823 ) dflash: rotate injected K/V cache when using K/V quantization Update src/models/dflash.cpp Co-authored-by: Georgi Gerganov ggerganov@gmail.com clearer format remove trailing whitespace Co-authored-by: Georgi Gerganov…
21 -
b10067
llama-quant : exclude i32 ffn_gate_tid2eid routing table from quantization ( #25787 ) DeepSeek-V4's ffn_gate_tid2eid tensor is an i32 token-id -> expert-id index table, not weights. It was never added to the name-based exclusion list alongside ffn_gate_inp.weight, so…
20 -
b10066
opencl: load and use kernel_gemm_moe_q6_k_f32_ns from bin kernel lib ( #25797 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU)…
19 -
b10064
opencl: transpose q4_K noshuffle scales for coalesced reads ( #25805 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu…
4 -
b10063
sync : ggml Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64 (Vulkan) Ubuntu…
37 -
b10061
tests : initialize all tensors in test_dsv4_hc to avoid NaNs in sentinel tensors ( #25822 ) Co-authored-by: Stanisław Szymczyk sszymczy@gmail.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
26 -
b10059
ggml-blas: default hadamard mul_mat to cpu routine ( #25710 ) Signed-off-by: Aaron Teo aaron.teo1@ibm.com Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64…
8 -
b10058
vulkan: Support Q2_0 ( #25430 ) vulkan: Support Q2_0 The backend perf tests for mat-vec-mul weren't very good at first (worse than q2_k), doubling the rows per workgroup made a big difference. reorder resolve merge conflict, adjust err threshold for f16->q2_0 set_rows Website:…
37 -
b10057
sycl: fix row calculation when K_QUANTS_PER_ITERATION is 1 ( #25690 ) sycl: fix incorrect row calculation when K_QUANTS_PER_ITERATION=1 Signed-off-by: Todd Malsbary todd.malsbary@intel.com sycl: use K_QUANTS_PER_ITERATION for non-reordered Q5_K kernel This is the only Q5_K…
24 -
b10054
docs: added a note about using OpenCl with Adreno 810 ( #25786 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
7 -
b10052
hexagon: L2 cache handling rework (dirty bit tracking with lazy flushing) and more MUL_MAT updates ( #25762 ) hex-mm: fix artificial limit in the solver that restricted number of act-prep threads hex-mm: fix warning hex-prof: do not apply --top to the timeline report hmx-mm: add…
18 -
b10051
kleidiai: Add SME vs SME2 distinction in kernel dispatch ( #25478 ) The current integration treats SME as a single capability (CPU_FEATURE_SME) with no distinction between SME(v1) and SME2. The kernels dispatched under CPU_FEATURE_SME use SME2-specific instructions, making…
10 -
b10050
vulkan: when using transfer queue for async copies, sync on event_wait to avoid race ( #25229 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
27 -
b10048
TP: fix Phi3, Bert, Plamo2/3, ChatGLM ( #25536 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
14 -
b10047
vendor: update BoringSSL to 0.20260713.0 ( #25624 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
24 -
b10046
tests: actually exercise test-recurrent-state-rollback ( #25758 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
32 -
b10045
server : allow text-only slot save/restore with mtmd ( #25076 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x…
25 -
b10043
CUDA: Support CUDA Virtual Devices ( #25228 ) support cuda virtual devices disable NCCL path when virtual devices are used label virtual devices in description; add GPUx2 server CI jobs code refactor Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple…
6 -
b10042
Enable CUDA graphs on volta+turing ( #25749 ) Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64…
35 -
b10041
server: Ignore empty / non-existing Origin headers ( #25756 ) Otherwise this gives lots of unnecessary warnings: W srv operator(): (CORS) skip non-localhost origin: Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
8 -
b10040
ggml-cuda : restore prop.integrated on HIP builds ( #24233 ) PR #16308 set info.devices[id].integrated = false unconditionally for all CUDA/HIP devices as a workaround for corrupted output on Jetson Orin ( #15034 ). On HIP/ROCm the device's real hipDeviceProp_t.integrated flag…
37 -
b10038
ci : add official website link to release notes ( #25728 ) Assisted-by: pi:llama.cpp/Qwen3.6-27B Website: https://llama.app macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU)…
4 -
b10037
quant : allow using manual tensor types with --pure ( #25716 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu…
32 -
b10036
opencl: disable FA and MoE weights repack to work around compiler issues for Adreno 850 GPU ( #25745 ) opencl: workaround for A850 compiler compat opencl: fix DX compiler version parsing and cleanup Co-authored-by: Li He lih@qti.qualcomm.com macOS/iOS: macOS Apple Silicon…
7 -
b10035
cuda: extract Q1_0 elements via __byte_perm ( #25628 ) macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux: Ubuntu x64 (CPU) Ubuntu arm64 (CPU) Ubuntu s390x (CPU) Ubuntu x64 (Vulkan) Ubuntu arm64…
18