llama.cpp releases
500 articles archived · Visit source ↗ · RSS
-
llama.cpp releases dev-tools 8d ago
b11452
model: support embeddinggemma2 (text+vision+audio) ( #30054 )
4 -
b11450: RPC: add `-sm tensor` (#26610)
rpc: allow -sm tensor fix flush for apple rdma move graph_uids to rpc_dispatcher cont : fix conflict cont: stop spinning dispatcher thread remove meta backend change add TODO to simplify logic rpc: bump major version Co-authored-by: Georgi Gerganov ggerganov@gmail.com
21 -
b11446
metal : fix excess threadgroup memory in quantized flash attention (#…
13 -
b11443
models : consolidate nextn row cropping into shared helpers ( #30017 ) mimo2 : always emit h_nextn the other nextn-capable models set it unconditionally Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-MOPD models : consolidate nextn row cropping into shared helpers replace the…
30 -
b11440
llama : re-reserve the sched when the nextn extraction flags change (…
14 -
b11439: ggml: refactor selective expert copying to user code (#29943)
ggml: refactor selective expert copying to user code tests: enroll two models into selective expert copy test tests: use deepseek2 as test model improve comment in ggml-backend.h Co-authored-by: Georgi Gerganov ggerganov@gmail.com cont: fix whitespace cont : better comments…
31 -
backup/allozaur-ui-shell-polish: ui : polish the shell
The secondary button gets its own look back and the chat add button its own light surface. Pressable elements get a pointer cursor again, the font rendering smooths on the app shell, the root layout resolves its props probe through the providers' api url, and the agent skills…
10 -
b11436: ggml-openvino: fix CI tests; fix GPU regressions. (#30037)
ggml-openvino: skip unselected graph branches and support DUP Upstream #29622 adds a mixed token/embd branch to every input embedding graph through ggml_build_forward_select(). Its nodes are not flagged for compute, but the backend translated them anyway, and the DUP in that…
33 -
b11434: hexagon: ssm-conv updates (#29971)
hexagon: ssm-conv double-buffered DMA for prefill and decode restructuring hex-ssm-conv: remove divs from loops and fix trace events hex-dma: improved SSM_CONV dma pipeline and streamlined dma_queue Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com
32 -
b11433: hexagon: add pool op support (#29995)
hexagon: add pool_2d support hexagon: add pool_1d support hex-pool: dma changes hex-pool: Optimize HTP pooling boundaries and DMA pipelining hex-pool: code cleanup and correctness fixes hex-pool: re-write the DMA pipeline hex-pool: pool chunking support hex-pool:…
19 -
v0.6.0
Overview llama.cpp v0.6.0 introduces the new llama_batch_ext extended batch API (with llama_process ) for mixed token/embedding inputs and MTP/deepstack state embeddings, adds support for the GLM-5.3-Flash (GLM5-Next) 320B hybrid model, the Clef decision model (text and vision)…
27 -
b11430: hexagon: matmul and flash-atten scalability updates (#29974)
hexagon: head-parallel flash_attn partitioning for row-split multicore In row-split mode each core computes its output row shard of every MUL_MAT, but flash_attn was previously partitioning by Q tokens (flat qrow split) instead of by heads. This forced every core to read the…
37 -
b11429: llama.cpp : bump version to 0.6.0 (#29997)
llama.cpp : bump version to 0.6.0 scripts : update summary prompt (#0)
16 -
b11424
vulkan: fix Flash Attention shmem write out of bounds ( #29988 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52864580 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64)…
16 -
b11425: CUDA: make the alloc_deps check batch independent (#29986)
Fixes #29980
20 -
b11422: cuda: use the vector lightning indexer kernel on MUSA (#29990)
cuda: stage the lightning indexer queries in head passes for MUSA MUSA archs 21 and 22 cap static shared memory at 28 KB, and the tile kernel staged the queries of all four heads next to the key tile for 33 KB. The queries are now staged in passes of…
10 -
b11414
vulkan: fix stale prealloc_y reuse across flash attention and soft_max ( #29591 ) Assisted-by: Claude Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52808438 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI…
28 -
b11412
llama : fix unexpected graph reallocation in the k-pool models ( #29958 ) llama : fix unexpected graph reallocation in the k-pool models Both k-pool models built a graph shape that depends on state the full-context reserve cannot know: qwen4exp branched on inp->cache_safe, which…
7 -
b11415: server: reject partial media truncation (#24076)
server: reject partial media truncation server: keep only the keep_first fix Drop the mtmd test helper change, which no longer builds since clip_image_f32_batch stores its entries by value, and drop the vision test: no test fixture reaches a cut between two adjacent media chunks…
9 -
b11408
ggml-cpu : add Q8_0 IME1 matrix kernel for SpacemiT X60 ( #28479 ) ggml-cpu : add Q8_0 IME1 matrix kernel for SpacemiT X60 On the SpacemiT X60, IME matrix acceleration only covered Q4_0/Q4_1/Q4_K. Q8_0 had no IME1 kernel, and since the SpacemiT build sets GGML_CPU_REPACK=OFF…
30 -
b11413: vulkan: sparse flash attention for quantized K/V (#29639)
vulkan: sparse flash attention for quantized K/V Assisted-by: Claude vulkan: single-scan sparse FA index compaction The compaction ran one workgroup per mask row and walked the row in BLOCK_SIZE chunks, with a workgroup scan per chunk. For decode that is one workgroup doing…
19 -
b11411
kv-cache: fix restoring mismatched KV cache rotation by saving exact …
33 -
b11407
vulkan : Fix undeclared identifiers when -DGGML_VULKAN_RUN_TESTS=ON (…
28 -
b11405
cuda: tile the lightning indexer over keys and tokens for 4 heads ( #2 …
26 -
b11404: metal : few-row MMA mat-mul (#29869)
metal : few-row MMA mat-mul and batched copies for speculative decoding Speculative decoding verifies a few draft tokens per step. Without the tensor API, Metal ran these mat-muls with the mat-vec kernels, whose time grows with every src1 row, so DFlash2 decoding on an M3 Ultra…
7 -
b11403: CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size (#29633)
CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size Signed-off-by: ynankani ynankani@nvidia.com adjust kernel selection logic Signed-off-by: ynankani ynankani@nvidia.com Co-authored-by: Johannes Gäßler johannesg@5d6.de
12 -
b11402
CUDA: prefer whole-tile FlashAttention scheduling for efficient two-s…
23 -
b11401
log, server: self contained colors, split child commands from logs in router mode ( #29895 ) log, server: make router child lines carry their own colors The logger writes the color reset after the trailing newline, so the reset opens the next line. On the shared pipe of a router…
12 -
b11400: llama: support both embd + raw tokens in batch (#29622)
llama: support both embd + raw tokens in batch add to test-llama-archs also check case llm_arch_supports_mixed_batch = false constant graph topology have dedicated input for mixed case rm set_tensor_backend is_embd --> type consolidate m-rope pos handling into one place nits
29 -
b11399: CUDA: refactor swizzling code (#29612)
CUDA: refactor swizzling code fix templates/loop bounds
26 -
b11397
cuda : move neu_padded to where it is used ( #29940 ) Signed-off-by: Adrien Gallouët angt@huggingface.co Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52636582 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64,…
37 -
b11396
ci : windows llvm build requires ninja multi-config ( #29959 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52619546 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
11 -
b11395: ci : add windows arm64 vulkan release (#29954)
add windows vulkan arm64 release add link
14 -
b11393
chat-peg-parser : clear current_tool when pending_tool_call is reset …
21 -
b11392
ci : set default permissions ( #29945 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52595372 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
13 -
b11391: cuda : move blocks_per_col to where it is used (#29939)
Signed-off-by: Adrien Gallouët angt@huggingface.co
17 -
b11390
CUDA: fix MMQ memory fault if n_expert >> n_ubatch ( #29941 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52581427 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS…
9 -
b11389
vulkan: fix rdna4 mat_vec tuning ( #29934 ) Website: https://llama.app Attestations: https://github.com/ggml-org/llama.cpp/attestations/52579133 macOS/iOS: macOS Apple Silicon (arm64) macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED macOS Intel (x64) iOS XCFramework Linux:…
9 -
b11388
imatrix: calculate activation-based statistics for new format (GGUF) imatrices ( #14891 ) Use activations to calculate the stats Determine calculation mode Compute entropy for activations Compute cosine similarity based on activations Compute l2 norm Add…
16