Built an open-source Cotypist/Granola hybrid: autocomplete on Gemma 4 E2B (~50 ms/suggestion on M4 Max) using your own meeting notes as context

I built a desktop app harness that keeps local meeting notes and can use them to autocomplete my typing as well as complement my dictation with additional context via reading my screen or using the previous meetings notes.

It's been a bit of a struggle to get it to work well and also to not be too intrusive and laggy. I've finally made it usable in the last few days so I'm sharing it here. I've done a lot of experiments with different models and quants and Gemma 4 E2B base at 6bit is the best balance of quality suggestions and low latency on a regular MacBook.

Head-to-head with Cotabby, 782 next-word checkpoints:

It's comparable performance wise to CoTabby, I'm still working on polishing the UX though.

GitHub: https://github.com/stevyhacker/lokalbot

If you try it I'd love to hear how it writes for you: which language, and how useful are suggestions. And if you know a small base model that beats Gemma 4 E2B at about 50 ms, I'll benchmark it.