llama.cpp releases · · 1 min read

b10699

Mirrored from llama.cpp releases for archival readability. Support the source by reading on the original site.

ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (#27960)

some backends (Metal, SYCL, WebGPU) require additional memory for
fleeting data for certain ops, which is reflected in their
get_alloc_size implementations.

add ggml_backend_op_alloc_size_may_expand() to the backend utils,
listing these ops, and assert in ggml_backend_buft_get_alloc_size
that a backend expanding the alloc size of a compute op only does so
for ops listed in the helper.

use the helper in the RPC backend to decide whether to query the
remote server for the actual alloc size, instead of a hardcoded list.

Assisted-by: pi:llama.cpp/Qwen3.8-27B

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from llama.cpp releases