LLM Inference Dashboard

Working on a resource dashboard, rich logs, lightweight 64mb cap, all local, scales on network API endpoints via collector, supports multiple engines (llama, strata, custom cuda engines, unsloth, LMS)

Not public yet, but curious if anyone would be interested?

It’s better logging and metrics then the default endpoint api provides. If you’re like me, you don’t just use 1 engine