b11400: llama: support both embd + raw tokens in batch (#29622)
Mirrored from llama.cpp releases for archival readability. Support the source by reading on the original site.
-
llama: support both embd + raw tokens in batch
-
add to test-llama-archs
-
also check case llm_arch_supports_mixed_batch = false
-
constant graph topology
-
have dedicated input for mixed case
-
rm set_tensor_backend
-
is_embd --> type
-
consolidate m-rope pos handling into one place
-
nits
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.