llama-doctor: CLI tool and live dashboard that reads your llama.cpp metrics and tells you what's wrong

Wait 5 sec.

https://preview.redd.it/93o5gjyuq9wh1.png?width=2800&format=png&auto=webp&s=86528bbbcee68e37ca775af011c1c16ec87eddd1 Points at a running llama.cpp or Unsloth Studio server, pulls /metrics, /props and /slots, and gives you severity-graded findings: prefill vs decode, queue pressure, prefix cache reuse, memory headroom. Exit codes map to severity, so cron can gate on it. Read-only, stdlib only, Python 3.10+. https://github.com/msalsas/llama-doctor Open to PRs, especially from anyone with a server that makes it crash.   submitted by   /u/msalsas [link]   [comments]