Spent an evening benchmarking mlx-serve against LM Studio on an M2 Max and hit something worth sharing, because it silently costs you most of the speedup. The gotcha: mlx-serve expects the MTP sidecar's tensors to be prefixed `mtp.` — `mtp.fc.weight`, `mtp.layers.0…`. The head published as `mlx-community/Qwen3.8-27B-MTP-4bit` (239 MB) ships them without a prefix: `fc.weight`, `layers.0…`. Drop it in and the server starts with no error, no warning, nothing in the log. It just quietly falls back to Prompt Lookup Decoding. How to tell you're affected — look for the spec stats line, not the startup flag: [spec-stats] mode=pld ← speculation is PLD, your MTP head is not loaded [spec-stats] mode=mtp attempts=31 accepts=67 per_draft_pct=69.8% depth=6 ← this is what you want Fix is three lines: import mlx.core as mx mx.save_safetensors(dst, {f"mtp.{k}": v for k, v in mx.load(src).items()}) Also: the file has to be at mtp/weights.safetensors inside the model dir, while HF publishes it as model.safetensors. A symlink farm works fine, no need to copy the weights. What it's worth (Qwen3.8-27B 4-bit, same weights, median of 5 runs, unique prefix per prompt so nothing is served from a prefix cache): prefill short/long decode short decode long mlx-serve, MTP actually loaded 49 / 145 tok/s 13.0 20.1 mlx-serve, no speculation 47 / 112 tok/s 8.0 11.2 LM Studio, same weights 59 / 101 tok/s 11.8 11.6 So +63 % / +79 % decode from the prefix alone. Note the second row: without speculation LM Studio's engine is actually faster here. MTP is what turns it around, which makes silently losing it expensive. One limitation I couldn't get past: MoE MTP heads fail to load at all — "Failed to load MTP sidecar: MlxError", with the prefix already correct. Both heads that loaded fine were dense and logged dense-mlp. So for Qwen3.6-35B-A3B I got nothing, while the dense 27B works. Filed an issue with the details. Environment: mlx-serve 26.9.5, MLX 0.32.2, macOS 26.6.2, M2 Max 96 GB. Numbers are decode/prefill measured client-side from the stream, not the server's own counters. Anyone got a MoE MTP head working on Apple Silicon, in any runtime?   submitted by   /u/Micha0827 [link]   [comments]