r/LocalLLaMA · · 1 min read

little tool for offline wikipedia RAG

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

little tool for offline wikipedia RAG

I was bored and handwrote a tiny 100-line bash script to let an agent search for and read articles from an offline wikipedia archive during a regular chat. It's not particularly useful, but it's definitely neat and a big step up from llama-cli. I got surprisingly good results even with LFM2.5-2B (Q8 at 0.5 temp), which is usually too weak to be of much use. (and Qwen had no trouble, obviously— see main image)

Just point it at your llama-server endpoint and chat away! Or whatever.

The handwritten writeup has more examples and a pretty comprehensive overview of how it works— I was surprised how simple the llama-server API is for general agentic stuff and toolcalling. Makes me want to make a proper harness...

edit: forgot to link github, but you should read the writeup instead 🙃

submitted by /u/mantisalt
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA