hey folks, can you try this out? i built an easy to understand LLM benchmark + tuner on top of NVIDIA's AIPerf. why .... running benchmark tools is easy, but figuring out what the numbers mean and what to do next is hard, especially if you're not a seasoned inference engineer. so this tells you how many users your server can handle at your latency targets, what breaks first, and which vLLM flag to change next it's in alpha .... would really appreciate feedback. github: https://github.com/S1LV3RJ1NX/kneebench sample report (llama 3.1 8b on one H100): https://github.com/S1LV3RJ1NX/kneebench/blob/main/docs/demo-run/kneebench-run-1.pdf   submitted by   /u/s1lv3rj1nx [link]   [comments]