b11600: graph: pad the KQ bias of the no-cache attention (#30446)
Mirrored from llama.cpp releases for archival readability. Support the source by reading on the original site.
The no-cache attention pads K, V and the mask to 256, but T5 also adds
a relative position bias of [n_tokens, n_tokens], so the add with KQ
fails unless n_tokens divides the padded width. Pad the bias with
zeros up to the KV dimension of the mask, like K and V.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.