llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 6d ago
b11500
hex-mmadd: do not assume aligned read/write when bias-add is fused ( #30133 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53932639 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
4 -
b11498
CUDA: fix norm family kernels when ne[2]/ne[3] exceed grid dim limits ( #28175 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53919465 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…
21 -
b11495
model : support classifier_activation for rerankers ( #29692 ) model : support classifier_activation for rerankers Assisted-by: Claude Opus 5.5 model : map classifier gelu to gelu_erf and accept tanh Assisted-by: Claude Opus 5.5 model : default act_cls to tanh, ModernBERT falls…
5 -
b11504
vulkan: extend sparse FA support to coopmat2 ( #30003 )
25 -
b11502
cuda : support arbitrary striding for unary ops on f16, f32, and bf16…
23 -
b11499
cuda : Use byte strides for roll to allow non-contiguous ROLL operati…
31 -
b11496
sycl: stage bulk uploads (model loading) through a pinned ring buffer…
25 -
b11493: sycl: add grouped MoE XMX GEMM (#29245)
Co-authored-by: cwriter cwriter@localhost
6 -
b11491: sycl: remove duplicate block-size defines from op headers (#29507)
All were duplicates of the values in ggml/src/ggml-sycl/presets.hpp
9 -
b11490
hexagon: enable alloc_buffer_n ( #30126 ) hex-bufs: add support for alloc_buffer_n hex-bufs: add support for splitting large tensors into separate buffers hex-bufs: update GGML_HEXAGON_MBUF to accept three values dyn,static,total hex-bufs: bump dyn. default to 512MB since 128MB…
6 -
b11489
hexagon: Q6_K weight dequant speedup ( #30121 ) hexagon: Q6_K weight dequant speedup Assisted-by: OpenCode unroll by another factor of 2 Assisted-by: OpenCode Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53807896 macOS/iOS: macOS…
23 -
b11487
hexagon: support tiled Q4_K and Q6_K GET_ROWS ( #30115 ) hexagon: support tiled Q4_K GET_ROWS Assisted-by: OpenCode properly reject Q4_K views Assisted-by: OpenCode hexagon: support tiled Q6_K GET_ROWS Assisted-by: OpenCode Website: https://llama.app Attestations:…
10 -
b11486
hexagon: improved GELU accuracy ( #30104 ) Assisted-by: OpenCode Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53797268 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
18 -
b11485
chat: fix jinja parser for TranslateGemma ( #30096 ) chat: fix jinja parser for translategemma log if missing soruce_lang_code or target_lang_code Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53794844 macOS/iOS: macOS Apple Silicon…
23 -
b11483
model : add LiquidAI/d1-3B decision model ( #30110 ) model : add LiquidAI/d1-3b decision model mtmd : read LFM2 image resize algo from GGUF Assisted-by: Claude Opus 5.5 common : rename decision type d1 to lfm2-d1 Assisted-by: Claude Opus 5.5 Website: https://llama.app…
10 -
b11482
cuda: FWHT kernels for block widths above 512 ( #29100 ) The CUDA FWHT covers widths 64 to 512. It runs one row per warp and keeps N/32 values per lane, so wider blocks need more registers per lane than that layout allows. fwht_cuda_block runs one row per thread block with 256…
20 -
b11488: model : add LiquidAI/d1-omni-600M decision model (#30114)
model : add LiquidAI/d1-omni-600M decision model Assisted-by: Claude Opus 5.5 mtmd : keep conformer GLU sigmoid on CUDA Assisted-by: Claude Opus 5.5 server : take d1omni audio through images and input_audio, scope memory-less lfm2 to non-causal Assisted-by: Claude Opus 5.5…
11 -
b11484: chat : name tool and argument parser rules by index (#30088)
bugfix: infinite recursion caused by a tool named 'call'( #29967 ) chat : use index for schema and argument rules tests : remove tests tests : add expect_rules to peg test parser Co-authored-by: Alde Rojas hello@alde.dev
11 -
b11481: mtmd: add cohere2 vision support (#30062)
cohere2 vision model address comments remove redundant mapping follow existing patterns fused linear_1
31 -
b11479: cuda: update uncoalesced memory reads in pool2d (#29425)
cuda: update uncoalesced memory reads in pool2d cuda: Added perf and restrict in pool2d Update POOL2D_WARP_KERNEL_MIN_WINDOW macro Co-authored-by: Georgi Gerganov ggerganov@gmail.com cuda: Fix compiler bugs Co-authored-by: Georgi Gerganov ggerganov@gmail.com
26 -
b11476
metal : few-row MMA mat-mul for the remaining src0 types ( #30065 ) The generic few-row MMA kernel works for any type with a 16-weight dequantizer, so it now also takes BF16, Q1_0, Q2_0, MXFP4, Q2_K, Q3_K, TQ2_0 and the IQ types. Each type starts at the row count where it beats…
20 -
b11477: llama: share the nextn tensor flags between models (#30097)
llama: share the nextn tensor flags between models Follow-up of the TODO in glm5-next: move the trunk-only and MTP-only detection that each model copied into a nextn_flags helper of llama_model_base. It probes the first trunk layer and the first NextN layer, and adds TENSOR_SKIP…
16 -
b11474
feat: add GLM5Next MTP, optimize ( #29928 ) llama : add GLM5-Next NextN (MTP) graph Build the GLM5-Next multi-token-prediction head as graph_mtp: the NextN block embeds enorm(tok)+hnorm(h) through eh_proj, runs one plain DSA layer and the shared lm_head, reusing the trunk's…
10 -
b11475
metal : fix MUL_MAT+ADD fusion when the residual is itself a MUL_MAT …
18 -
b11471
vocab : add plamo fim tokens ( #30090 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53567889 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
10 -
b11469
musa: use the tile lightning indexer kernel ( #30080 ) Signed-off-by: Xiaodong Ye xiaodong.ye@mthreads.com Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53563400 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
27 -
b11468
vendor : update cpp-httplib to 0.60.0 ( #30081 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53557265 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
17 -
b11466
ggml-webgpu: fix flash_attn supports_op check for overlapping KV ( #28205 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53548008 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS…
17 -
b11465
tests : retain the anchor when testing recurrent rollback ( #29923 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53541820 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel…
38 -
b11463
sycl: accelerate GLM MLA prefill with MKL flash attention ( #29171 ) sycl: accelerate GLM MLA prefill with MKL flash attention GLM-4.7 Flash uses an MLA shape with 576-wide Q/K heads, a 512-wide V head, GQA 20, and F16 KV. The SYCL dispatcher rejects this shape because the…
13 -
b11472
sampling : use greedy selection for eligible temperature-zero chains …
31 -
b11467: imatrix : include clocale for std::setlocale (#30079)
Co-authored-by: yarik dazzywi@github.com
26 -
b11461
vulkan: fix amd iGPU slow checkpoint read ( #30049 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53465294 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
32 -
b11460
sycl: add IQ3_S multi-column MMVQ ( #29500 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53455527 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework…
10 -
b11459
ggml-webgpu: no dawn native features on wasi ( #27069 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53445261 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
9 -
b11458
hexagon: CPY/CONCAT/CONT/DUP overhaul to use DMA/HVX for all cases (#…
10 -
b11457
cuda : add BF16 support for XIELU ( #29955 ) The XIELU CUDA kernel template is already generic over the element type; only the F32/F16 type assertion and the else-if dispatch were missing. Add the nv_bfloat16 branch to the launcher, and drop the temporary supports_op gate in…
10 -
b11455
vocab : implement PLaMo-3 tokenizer pre-segmentation ( #30045 ) vocab : implement PLaMo-3 tokenizer pre-segmentation The PLaMo-3 tokenizer inserts hard boundaries before running the Unigram DP, around <|plamo:...|>-looking text, and around runs of at least 4 identical characters…
6 -
b11454
model : add K2 Horizon dense and MoVA support ( #29535 ) model: K2 Horizon gguf conversion code model: loading hparams and tensors in k2-horizon.cpp model: K2 Horizon compute graph model: K2 Horizon compute graph adjustment and registering tokenizers model: K2 Horizon chat…
4 -
b11451
opencl: fix OOB read in adreno xmem GEMM ( #30041 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/53310534 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
9 -
b11449
ggml: fix CLAMP on non-contiguous views (CPU, CUDA) ( #29517 ) ggml: fix CLAMP on non-contiguous views (CPU, CUDA) CUDA clamped ggml_nelements values flat and ignored the view strides. CPU addressed row j as j*nb01 and ignored nb02/nb03. Both now follow the strides of dims 1..3;…
7 -
b11448
cuda: BF16/FP16 conversion to f32 chunking ( #29442 ) ggml-cuda: chunk large BF16/FP16 to F32 conversions Update ggml/src/ggml-cuda/ggml-cuda.cu Co-authored-by: Johannes Gäßler johannesg@5d6.de Update ggml/src/ggml-cuda/ggml-cuda.cu Co-authored-by: Johannes Gäßler…
12