tandem.cpp is a patch set for llama.cpp that makes MoE models that don't fit in VRAM run faster. It keeps the most-used experts on the GPU and lets the CPU help with the rest, for any MoE model llama.cpp supports. Single-user decode is the target. Consumer PC (i5-12400F, 64 GB DDR4, RTX 4090), GSM8K, decode tok/s: Model llama.cpp tandem.cpp Strata Qwen3.8-Flash-Next IQ3_S + MTP, greedy 29.8 97.3 (3.3×) 103.6 same, Qwen's default sampling 29.4 96.1 (3.3×) 98.1 gpt-oss-120b, greedy 27.8 51.5 (1.85×) — gpt-oss-120b, recommended sampling 27.7 48.0 (1.73×) — Tuned for Qwen3.8 only, but other MoE models that don't fit in VRAM gain too: typically 1.4–2.1× (gpt-oss-120b, GLM-4.5-Air, Qwen3.6-35B-A3B, DeepSeek-V4.1). Prompt processing is 1.4–1.8× faster on a 64k prompt. Strata is shown in its exact mode (every routed expert computed, like llama.cpp). Its default skips low-ranked experts, which is faster but approximate (127.1 here). Independent project, not affiliated with ggml-org. Build, run commands, and full results on 4 machines: https://github.com/ewhacc/tandem.cpp   submitted by   /u/smflx [link]   [comments]