hey, my name is Ye, I vibecode at Nace.ai and we do a lot of work with enterprises, and a lot of the time we're inside their networks with no internet access and outdated cloud hosted models. considering we work on macbooks, we have built Kapitsa, an open-source (Apache 2.0) inference engine for Apple Silicon, to run coding agents on local models that would actually fit on your Macs. Kapitsa runs only two model families: Qwen (3.5 4B, 3.8 27B, 3.6 35B-A3B, and 3.8 Flash-Next on 128 GB Macs) and NVIDIA Nemotron (3.5 Lightning, Bolt and Super-VL, plus Dory 4B (this is new, we didnt test Dory too much yet). focusing on only these models allows us customize the Metal kernels for each one on Apple Silicon GitHub: https://github.com/nace-ai/kapitsa - It's open source, and Apache 2.0. Quickstart brew install nace-ai/tap/kapitsa # or: curl -fsSL https://raw.githubusercontent.com/nace-ai/kapitsa/main/install.sh | bash kapitsa opencode --model 35b # or kapitsa claude, kapitsa codex, etc Decode Benchmarks (we at nace dont like benchmarks, DO NOT trust them) instead we have built a quite simple cli for you to be able to bench it yourself. please submit your results, we are looking for contributors for this project ^^ kapitsa bench MacBook Pro M4 Pro (2024), 48 GB: Qwen Models tok/s Kapitsa mlx-serve oMLX MTPLX Ollama MLX llama.cpp LM Studio Rapid-MLX Qwen3.8 27B 63 54 45 36 32 28 18 15 — Qwen3.6 35B-A3B 154 124 127 80 89 86 69 86 — Qwen3.5 4B 179 137 102 104 109 88 70 81 98 MacBook Pro M4 Pro (2024), 48 GB: Nemotron tok/s Kapitsa oMLX MLX llama.cpp Nemotron 3.5 Bolt 74B-A9B 64 35 32 ✗ Nemotron 3.5 Lightning 30B-A3B 106 — 65 — Nemotron Dory1.0 4B 64 ✗ 48 6 (LOL) MacBook Pro M5 Max (2025), 128 GB: Qwen tok/s Kapitsa oMLX Ollama LM Studio MTPLX MLX mlx-serve llama.cpp Qwen3.6 35B-A3B 272 232 166 127 123 112 — — Qwen3.8 27B 115 — 68 32 69 30 — — Qwen3.5 4B 294 — 213 154 175 135 — — Qwen3.8 Flash-Next 180B-A7B 105 107 60 ✗ 88 77 108 96 Flash-Next is the one model where Kapitsa isn't fastest: mlx-serve and oMLX are ahead by 2-3%. (our leadership team told us that they will buy us 128gb to optimize flash next, if this thing gets 1k stars xd) MacBook Pro M5 Max(2025), 128 GB: Nemotron tok/s Kapitsa oMLX mlx-serve llama.cpp Ollama MLX MTPLX LM Studio Lightning 30B-A3B 239 190 190 159 153 139 131 110 Super-VL 120B-A12B 85 52 — 66 64 46 — 61 Bolt 74B-A9B 127 — ✗ ✗ ✗ — ✗ ✗ MacBook Pro M1 Pro, 16 GB (2021) tok/s Kapitsa mlx-serve oMLX Ollama LM Studio MLX MTPLX llama.cpp Qwen3.5 4B 85 62 55 55 47 46 42 38 Nemotron Dory1.0 4B 40 — ✗ — — 30 — 4 Where the speed comes from 1) custom Metal kernels per architecture (Gated DeltaNet + attention for Qwen3.5-family, Mamba-2 + attention + LatentMoE for Nemotron-H) and per weight format (MLX affine, NVFP4, NVIDIA's 2-bit experts). 2) speculative decoding: a draft model proposes a block and the model checks it in one pass (DFlash2 for the Qwen 27B and 35B, z-lab's DFlash for the 4B, DSpark for Lightning, NVIDIA's LiLiCorr for Bolt and Super-VL). 3) caching that works with hybrid models. Their recurrent layers break a plain KV prefix cache, so each conversation keeps KV plus recurrent-state checkpoints at turn boundaries, and the engine reads your last answer into the cache while you read it. 4) on the 27B and Bolt, part of each layer's prefill runs on the Neural Engine alongside the GPU. (biggest boost) 5) requests in flight at once (an agent's subagents, opencode's title request) share GPU passes instead of queueing. we also made sure that speculation doesn't change what the model writes: greedy keeps the longest drafted prefix the model itself would pick, sampling is basically like standard speculative sampling, so tokens come from the models own distribution. quality drops a bit since its the quantized 4-bit weights, the as the other MLX engines. although you can pick it with --model 27b:6bit if need be, we reccomend using 4bit, it does the job CLI brew install nace-ai/tap/kapitsa kapitsa doctor # which models and context sizes fit this Mac kapitsa opencode --model 27b # or: kapitsa pi / claude / codex kapitsa serve spins up a server that works with your claude,codex etc, or any custom requests. kapitsa bench runs the same benchmark on your Mac, and kapitsa bench --submit adds it to the community results (it shows the JSON and asks first). M2 and M3 Macs, and 16-36 GB ones, are the results we want most. (We need a wide variety of devices) why Qwen and Nemotron We picked the models people here run for local coding and wrote kernels for exactly those architectures. Other Qwen3.5-family checkpoints load from Hugging Face, a folder or a GGUF; other architectures don't, and adding one is real work. [What's next, if you want to say: Qwen 4 on release, Flash Next] Things to mention Apple silicon only: M1 or newer, macOS 14+ Lightning and Dory are text only. New code, so expect bugs. Issues welcome, we have quite a bit of fable credits dedicated to this project ^^. Credits z-lab (DFlash) -awesome ideas, DeepSeek (DSpark), Inco(DFlash2) - cool, Qwen (MTP heads), NVIDIA (LiLiCorr and the Nemotron checkpoints), MLX(quantization), mlx-serve(amazing) Fable 5.1, Elorian Glimpse, Opus 5.5, Nace.AI coding models Asks try it yourself and upload your benches please. its a new codebase, bug reports/ suggestions are welcome.   submitted by   /u/DotaMate [link]   [comments]