I Built a O(NlogN) attention system that retains 97% accuracy over long context (MQAR)[P]
Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.
| | ALHR- Adaptive learnable Hierarchical Routing is a static binary tree based system that uses learnable functions to REDUCE the amount of keys used. It takes less memory and SCALES much better VRAM with tokens [link] [comments] |
More from r/MachineLearning
-
I turned my dog Jackies Brain into a working AI [P]
Oct 14
-
Event2Vec: additive event embeddings where a linear score on a history decomposes exactly into per-event terms [P]
Oct 14
-
Binary noise classification using CRNN mean teacher algo [R]
Oct 14
-
Localized Neural learning vs backpropagation [D]
Oct 14
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.