Came across this popular open source inference engine Tensorfold https://github.com/ashhart/TensorFold Using their official Vontra/Qwen3.8-27B-MLX-4bit with drafting model z-lab/Qwen3.8-27B-DFlash2, I can reach 40-60 tps on mac mini m5 pro. AGAIN, it is PRO on mac mini, not even ultra studio. For me, it is the first time (on mac ecosystem) that an inference engine to beat MTPLX. I have tested omlx, dflash2, mlx, llama-cpp, lm-studio, unsloth in the past few months, and none of them come close to MTPLX (running Qwen 3.8 optimised for speed, roughly 4bit?) The more exciting part is that this enables me to seriously consider about replacing my RTX-3090ti with this mini running tensorfold as the main inference server setup. That old 3090ti, running ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF with MTP IQ3_S (12.1 GB), reaches 50-70 tok/s, which is, in my opinion, similar to the 40-60 tok/s I achieve with mac mini. The only one caveat is htat the 3090ti still has like 4x faster prefill than mac mini. Spec: M5 pro, Mac mini, 64gb, 1TB SSD Testing harness: pi coding agent without any packages install yet. Model: Qwen3.8-27B 4bit What's your thoughts on Tensorfold?   submitted by   /u/AdRepulsive7837 [link]   [comments]