If I wanted to locally compare performance for multiple different offline LLM models across various modalities (speed, efficiency, reasoning, tool use, programing/coding, etc), what strategies, prompts, software are commonly used to do this? There are so many models to choose from and run, I'm trying to determine things like whether slower but more complex models are worth running over slightly faster ones, how models that run at similar speed compare to each other's capabilities, and whether any of the custom engines that have been popping up other than llama.cpp/strata are worth it. Does this sort of thing exist as something that can be ran locally, and if so, how?   submitted by   /u/insomniating [link]   [comments]