Prefill: 150 tk/s Decode: 25 tk/s Its neither fast or power efficient but surprisingly useful for a local LLM setup. Thanks to Strata. All the experts fit on the VRAM. I'm only using strata to maximize prefill speeds. No code optimization done. Simply lifted the nvidia architecture to on strata to allow maxwell cards.   submitted by   /u/jmuff98 [link]   [comments]