llama.cpp releases · · 1 min read

b11574: hexagon: optimize CPY/CONCAT — round 3, short-rows edition (#30341)

Mirrored from llama.cpp releases for archival readability. Support the source by reading on the original site.

  • hexagon: move the converting CPY in blocks of rows per DMA descriptor so short rows are not DMA-latency bound

  • hex-cpy: cleanup cpy cursor handling

  • hex-cpy-concat: introduce threaded dma/hvx gather-row helper

Co-authored-by: Ruslan Karymov 67752306+karusrus@users.noreply.github.com

  • hex-cpy: remove leftover rejections for extended buffers

  • hex-cpy: relax the gate for fast-path

  • hex-cpy: remove scalar divs and vtcm access and use explicit gather_sync

  • hex-cpy: add proper support for mdev work splitting to fastpath

  • hex-cpy: optimize dense gather-row tasks

  • hex-cpy: unroll gather-rows loops

  • hex-cpy: add fastpath for f16

  • hex-cpy-concat: support for colapsing 3/4D dims

  • hex-cpy: double-buffer support

  • hex-cpy: factor out gather-rows into more of an op by itself

  • hex-cpy-concat: dedup kernel-params

  • hex-cpy-concat: step-down nr to fit instead of hard fail

  • hex-gather: replace div with a mask

  • hex-gather-rows: cleanup dma pipeline and remove the stalls

  • hex-gather: specialize compute funcs

  • hex-gather: reduce scalar reg.spills and func calls

  • hex-gather: do not recompute rows-per-thread on the npu

  • hex-build: add support for building/installing just hexagon

  • hex-cpy-concat: remove redundant barrier in concat and move cpy barrier under fence

  • hex-gather: annotations for dma-out

  • hex-mdev: make all key vtcm_acquire sync state atomic (fixes random hangs in mdev setups)


Co-authored-by: Yiwei Shao yiwei@aizip.ai
Co-authored-by: Ruslan Karymov 67752306+karusrus@users.noreply.github.com

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from llama.cpp releases