Prismix
llama.cpp releases · · 1 min read

b11480: llama : add a GPU cache for MoE experts kept in host memory (#29887)

#model-release #gpu

Mirrored from llama.cpp releases for archival readability. Support the source by reading on the original site.

Like Read original ↗
  • llama : add a GPU cache for MoE experts kept in host memory

Assisted-by: Claude

  • use llama_moe_cache_ptr

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from llama.cpp releases