I wanted to know how far one prosumer-class card goes with a big mixture-of-experts model, and what you pay for it. Setup: one RTX PRO 4500 (32 GB), 120 GB RAM, Ryzen 9 9950X3D, Linux. Model: DeepSeek V4.1 Flash 1017 as a 2-bit GGUF (ISTA DASLab's GSQ-RCO IQ2_XXS), 232 GiB on disk. The experts stay in RAM, the busiest ones sit on the GPU, and DeepSeek's own DSpark draft model proposes five tokens that the big model checks. What I measured: 49.9 tok/s decode with the draft model (median of three runs), 37.2 without. The draft does not change the output. Against the original weights the 2-bit quant has KL divergence 0.248 and top-1 agreement 83.5 %. That is the real cost, and comparing it to another quant instead of the original hides it. With thinking on and a 16k context, long thinking ends hard. A thinking budget of 8000 lets my two test tasks finish. At 64k without a budget both tasks thought 60,000 tokens and never answered. Very long thinking can fall into a soft loop of near-identical sentences. Speed is bound by RAM speed here (DDR5 at 4800, four sticks). I would like numbers from faster memory. If you have a 16 or 24 GB card: the 125B Qwen3.8-Flash-Next runs the same way from 16 GB and 64 GB RAM at about 15 tok/s, 91 on the 32 GB card. Disclosure: I wrote the engine fork (FLE, MIT) and the launcher (FLE Link, MIT) this runs on. Engine: github.com/sirfyyn/FastLocalExperts. The launcher and the per-card tables: flelink.run. Happy to run a prompt of yours and post the output.   submitted by   /u/Ryyn_- [link]   [comments]