llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 14h ago
b11618: HIP: use 64-row MMQ tiles for some quant types on RDNA3.5 (#28195)
The RDNA3.5 config table selects a 64-row / 128-thread tile up to J=32 and a 128-row / 256-thread tile above that. On gfx115x the smaller tile leaves enough LDS free for a second workgroup per CU, while the larger one does not. Extend the 64-row tile to the (type, J) pairs where…
33 -
b11614
vulkan: Define the shader count for Intel Arc Pro B65 + B70 ( #30431 )
23 -
b11613: vulkan: support mulmat bf16 activations (#29069)
vulkan: support bf16 src1 in mul_mat clean up mul_mat b-type selection fix conditions
15 -
b11612: unicode : add opt-in NFC normalization for Unigram tokenizers (#30390)
unicode : add opt-in NFC normalization for Unigram tokenizers Note: used help from Codex, Sol 6.1 Medium Fixing initial bot comments + linter error (using existing out instead of print) removed test_nfc_tokenizer
12 -
b11610
Enable OpenSSL in FP32/FP16 SYCL builds to restore HF HTTPS downloads…
17 -
b11608
server : escape the special tokens without std::regex ( #30243 ) The d1-omni and kev prompts escape "<|name|>" in the request text with std::regex_replace. libstdc++ recurses for each char of the name, so a state of "<|" and 30000 letters overflows the stack of the HTTP thread…
32 -
b11607: chat: replace output string with parts (#30389)
chat: replace output string with parts reserve first
34 -
b11606
ggml : simplify backend search path error handling ( #28367 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55616656 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon…
24 -
b11604
kleidiai: add SME2 Q4_K and Q6_K matmul support ( #29219 ) Change-Id: Ib4aa4c9814c010f9cf084783cc80612e364f9a27 Change-Id: I406a6606c3fea51f8cd32331c1ed71466845d363 Change-Id: Ib4aa4c9814c010f9cf084783cc80612e364f9a27 Signed-off-by: Charles Xu charles.xu@arm.com Website:…
19 -
b11602
meta: recreate all subgraphs when the arena is reallocated ( #30453 ) meta: recreate all subgraphs when the arena is reallocated chore: revert added test cases Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55603876 macOS/iOS: macOS…
31 -
b11601
hexagon: add configurable HTP DCVS/HMX power mode ( #30423 ) hexagon: add configurable HTP DCVS/HMX power mode hexagon: address PR review feedback (drop CLI flag, align QAIRT power modes, comment/log cleanup) hexagon: add --hex-power-mode flag to run.py Hooks…
5 -
b11598
model: load YaRN beta metadata for laguna models ( #30331 ) Assisted-by: Codex Co-authored-by: Adam Caldwell accaldwell@users.noreply.github.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55581133 macOS/iOS: macOS Apple Silicon…
9 -
b11594
vulkan: allow multiple devices to access gpu-visible host memory ( #29741 ) vulkan: allow multiple devices to access gpu-visible host memory clean up erroneous newlines add Windows support handle nullptr fix enable condition Website: https://llama.app Attestations:…
26 -
b11600: graph: pad the KQ bias of the no-cache attention (#30446)
The no-cache attention pads K, V and the mask to 256, but T5 also adds a relative position bias of [n_tokens, n_tokens], so the add with KQ fails unless n_tokens divides the padded width. Pad the bias with zeros up to the KV dimension of the mask, like K and V.
11 -
b11599: llama : do not map foreign ranges into a backend buffer (#30405)
llama : do not map foreign ranges into a backend buffer A buffer created from a host pointer pins everything that it wraps. When several contexts share the same file (lazy tensors, MoE tensors kept on the CPU with -cmoe), the single buffer that spans the first and the last…
34 -
b11597: graph: skip the LM head for pooled embeddings (#30445)
graph: skip the LM head for pooled embeddings A context that extracts pooled embeddings outputs one vector per sequence, but every model still expanded its LM head, so the graph computed n_vocab logits per token and the context kept a host buffer for them. The models now only…
31 -
b11592: cuda : add GGML_CUDA_GRAPH_DEBUG and read env vars once (#30327)
ggml-cuda: gate CUDA graph debug logs behind GGML_CUDA_GRAPH_DEBUG ggml-cuda: cache env var lookups in static const instead of re-reading per call ggml-cuda: read env vars exactly once via ggml_cuda_env_flag/ggml_cuda_env_int ggml-cuda: consolidate env flag reads into single…
21 -
b11590
vulkan : split matmul dispatch when it would exceed maxComputeWorkGro…
10 -
b11589: spec : remove tokens assert from dsv4 graph_mtp (#30444)
cont of #29622 , where we forgot to remove token assert
15 -
b11588: model : allow Qwen3.8 Flash Next to accept image embds for MTP (#30397)
Cont of #30257 , closes #30307
26 -
b11586: cuda : warnings cleanup (#30420)
cuda : warnings cleanup Signed-off-by: Adrien Gallouët angt@huggingface.co Update ggml/src/ggml-cuda/common.cuh Co-authored-by: Sigbjørn Skjæret sigbjorn.skjaeret@huggingface.co Signed-off-by: Adrien Gallouët angt@huggingface.co Co-authored-by: Sigbjørn Skjæret…
25 -
b11584
llama : skip logits buffer allocation when graph produces no logits (…
30 -
b11583
graph : avoid reallocations in the "no cache" path ( #30439 )
27 -
b11581
models : remove redundant ggml_cont calls now supported by all unary kernels ( #30198 ) models: remove redundant ggml_cont calls now supported by CUDA unary kernels models: restore cont_2d after sigmoid for broadcast mul, clean dflash cont Update src/models/qwen3next.cpp…
14 -
b11579
ci : build OpenVINO releases with GGML_NATIVE=OFF and CPU backend var…
10 -
b11577
model: fix vllm d3 multi seq case ( #30408 ) model: fix vllm d3 multi seq case add TODO Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55219202 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled)…
22 -
b11576
graph: pad the KV dimension of the no-cache attention to 256 ( #30416 ) The no-cache path builds a [n_tokens, n_tokens] mask and passes K and V unpadded, so FLASH_ATTN_EXT sees K->ne[1] == n_tokens. The CUDA kernels for head size 512 require a multiple of FATTN_KQ_STRIDE, so the…
37 -
b11575
metal: fix metal tensor kernel_mul_mm error for large tensor ( #28748 ) metal: fix metal tensor kernel_mul_mm error for large tensor Assisted-by: Codex Update ggml/src/ggml-metal/kernels/mul_mm.metal Co-authored-by: YiChen Lv 63285796+forforever73@users.noreply.github.com…
21 -
b11573
vulkan: pack FMA for f16 accumulation, no dot product op ( #29877 ) vulkan: pack FMA for f16 accumulation, no dot product op vulkan: dot_product packs two rows per f16 FMA vulkan: rename dot_product to dot_product_v2 Website: https://llama.app Attestations:…
13 -
b11574: hexagon: optimize CPY/CONCAT — round 3, short-rows edition (#30341)
hexagon: move the converting CPY in blocks of rows per DMA descriptor so short rows are not DMA-latency bound hex-cpy: cleanup cpy cursor handling hex-cpy-concat: introduce threaded dma/hvx gather-row helper Co-authored-by: Ruslan Karymov…
37 -
b11571
model: support vllm decision 2 model ( #30378 ) models: support pplx-decider vllm d2 update clarify naming rule stricter validation fix ci rm redundant filter_tensors Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/55070168 macOS/iOS:…
10 -
b11570
tests : link each test against only the libraries it uses ( #30299 ) Give llama_build() and llama_build_and_test() a LINK keyword (default: llama-common) so call sites can declare what a test actually needs: ggml-only tests now link just ggml: test-opt,…
37 -
b11572: model: support vllm d3 models (#30383)
models: support pplx-decider vllm d2 update clarify naming rule stricter validation d3
28 -
b11569: metal : fix mul_mat_id with duplicate ids (#30373)
metal : fix mul_mat_id limits and kernel name metal : fix mul_mat_id with duplicate ids kernel_mul_mm_id_map0 assumed at most one match per token and a fixed capacity of ne21 entries per expert, which produced wrong or dropped output rows and could overflow into the slice of the…
27 -
b11568: examples : fix draft sampling and rejection handling (#30095)
examples : fix draft sampling and rejection handling This commit fixes two issues in the speculative example which are described in detail in the linked issue. Resolves: #29975 examples : add kv-unified requirement for parallel > 1
25 -
b11564
vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size ( #30164 ) vulkan: specialize quant block size for CM2 matmul vulkan: use ternary expressions to avoid switch lowering for CM2 quant block size vulkan: clarify CM2 quant block size comment Website:…
15 -
b11563
ggml: Change default visibility of symbols to hidden ( #30014 ) See discussion in #29997 . This prevents a handful of internal symbols leaking via GGML_API. Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/54949054 macOS/iOS: macOS…
35 -
b11562
vulkan: fix buffer logging on 32-bit platforms after source split ( #3 …
30 -
b11560
hexagon: add IM2COL_3D ( #30305 ) hexagon: add IM2COL_3D support Assisted-by: OpenCode hexagon: add DMA paths for IM2COL_3D Assisted-by: OpenCode hexagon: avoid scalar fp16 conversions in IM2COL Assisted-by: OpenCode remove in-loop spills in pre-v81 devices Assisted-by: OpenCode…
33 -
b11559: opencl: improve fa, allow dk512 for gemma-4, improve dk64 (#30266)
opencl: enable Gemma-4 E4B GPU decode Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com opencl: extend Gemma-4 GPU decode to E2B Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com opencl: optimize DK64 GQA8 decode Co-authored-by: Hongqiang Wang wangh@qti.qualcomm.com…
17