hey folks, I'm Ye, one of the contributors to open-source project called Kapitsa, Apple Silicon Inference engine optimized specifically for Apple Silicon to run Qwen, Nemotron, Flint and Clover models locally. Our custom-written Metal kernels let us tune for each chip and each model. https://github.com/nace-ai/kapitsa/ The latest update, 0.2.26, is out. Since our last M5 Max numbers, Flash-Next 180B reads prompts 46% faster and the 27B writes 8% faster, and DiffuSpace's Clover 35B joins the catalog. Kapitsa also runs Qwen Flash-Next 180B with SSD offloading: on a Mac that can't hold it, the experts that don't fit in RAM stream from the SSD, so it runs on a 48 GB MacBook Pro at 61 tok/s. Here is the showcase on a 2026 MacBook Pro 16 with an M5 Max and 128 GB. We ran six models from 1K context all the way up to 32K, against the next-fastest engine for each model on the same Mac. Try it (any Apple Silicon Mac; models in this post: 35b, 27b, flash-next, bolt, flint, clover): brew install nace-ai/tap/kapitsa kapitsa serve --model 35b # OpenAI and Anthropic APIs on localhost kapitsa opencode --model 35b # or claude, codex, pi kapitsa bench # benchmark your own Mac Results https://preview.redd.it/0zmlaxm2x2zh1.png?width=2400&format=png&auto=webp&s=5f56c7445044fb825fa218b280e85d0307bc2d0b Decode is a 512-token coding answer, prefill a 4K prompt, median of 3 runs: Model Decode, tok/s Prefill, tok/s Qwen3.6 35B-A3B 346 6,036 Clover 35B-A3B 304 5,981 Flint 40B 265 5,770 Nemotron Bolt 74B 154 1,075 Qwen3.8 27B 143 904 Qwen Flash-Next 180B 108 1,701 From 1K context all the way up to 32K. At 32K the 35B still writes 175 tok/s, and Flash-Next barely drops (90 to 78): Prefill from 1K to 32K context Decode from 1K to 32K context Against the other friends Against the next-fastest engine for each model on the same Mac: 1.23x Splash on the 35B, 2.2x oMLX on Bolt, 1.18x rec on Flint and 1.12x mlx-serve on the 27B. Flash-Next is a tie with mlx-serve, Splash still reads prompts faster on the two Qwens, and nothing else runs Clover on a Mac. Decode against the other Mac engines Every cell with versions and flags: benchmarks. Raw runs: results. Why it's fast: KFlash Other engines check the drafter's guesses as one chain and stop at the first wrong one. KFlash checks a tree of the drafter's best guesses in the same pass, so more tokens survive each pass. How KFlash works: the drafter's chain against KFlash's tree On Apple GPUs that's nearly free: the GPU works in 8-row tiles, and a one-token step already pays for 8. Every token it keeps is one the big model would have written itself, so the output doesn't change, only the speed. How KFlash works shoutout to splash, ollama, omlx, llama cpp and the folks who Send us your benchmarks Run the same test on your Mac and add it to the community results. It takes 3-15 minutes: brew install nace-ai/tap/kapitsa brew install gh && gh auth login # results are sent as a GitHub pull request kapitsa bench --submit # pick a model; it asks before sending anything Plug in and close heavy apps while it runs. At the end it asks once: Enter sends the result as a pull request from your GitHub fork, and n keeps it on your Mac. What it sends is about 2 KB: your Mac's model, chip, RAM, macOS version and the numbers, with no serial number, user name or paths (add --dry-run to see exactly what it would send). No GitHub? Paste the numbers it prints in a comment here. BY THE WAY This is a new codebase so, sometimes there could be some weird prefill/decode bugs.   submitted by   /u/DotaMate [link]   [comments]