Lemonade end-of-summer project update, now serving 15 engines!

Hi everyone, it's been a while since I posted so here's an update on what the Lemonade community has been up to this summer. Our overall mission is to enable local AI builders with everything they need to make great apps and agents, while keeping the stack turnkey, portable, and modular.

What you get is a single install on any OS that runs as a service and manages a wide array of models and engines behind a single base URL, and ties them all together with an advanced router. The same capabilities are available in an embedded SDK that can be bundled into apps.

Here are some highlights from this summer:

1. Cross-platform support has come a long way this summer, with CUDA, ARM64, Metal, and Vulkan backends for all core engines.
2. Lots of experimental engines added to enable new modalities (music, 3D assets, etc.) and new optimizations (DwarfStar4, TheNoise).
3. The router now supports semantic and policy routing, allowing automatic on-the-fly LLM selection based on the prompt.

In terms of roadmap, we are hard at work on a complete replacement for the GUI (entering beta testing soon), as well as benchmarking tools and a plugin interface to help us all make sense of the rapid pace of new engine and backend releases we're seeing.

Lemonade is a community-driven project and anyone can help define the roadmap and feature set. Most of the work here is not mine, huge shoutout to u/mikkoph u/ilintar u/Geramy and the 100+ contributors making this a reality.

GitHub: https://github.com/lemonade-sdk/lemonade

Discord: https://discord.gg/5xXzkMu8Zk