b11567: llama : report compute graph reallocations during inference (#30329)
Mirrored from llama.cpp releases for archival readability. Support the source by reading on the original site.
Add an optional ggml_backend_sched_realloc_callback that user code registers with
ggml_backend_sched_set_realloc_callback to be notified when the buffers reserved for a
graph are not sufficient and have to be reserved again. The callback reports whether the
backend assignments and the graph size changed since the previous scheduling.
llama_context registers the callback when creating the scheduler (wrapped into the new
llama_context::sched_new helper) and treats the reallocations that follow a change of the
backend assignments as expected, counting the rest by graph size - unchanged or changed
from the previously scheduled graph. Both counters are reported on one line in the perf
stats.
Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.