b11403: CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size (#29633)
Mirrored from llama.cpp releases for archival readability. Support the source by reading on the original site.
- CUDA: use MMVF for thin f16/bf16 mul_mat at small batch size
Signed-off-by: ynankani ynankani@nvidia.com
- adjust kernel selection logic
Signed-off-by: ynankani ynankani@nvidia.com
Co-authored-by: Johannes Gäßler johannesg@5d6.de
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.