Not long ago, DFlash2 for Qwen3.8-Flash-Next was released, but initially llama.cpp compatibility wasn't there. Now it is. I quantized my own GGUFs of the drafter and tested it out in llama.cpp v0.8.0, the first release that supports DFlash2 for FN. Speed: MTP (acceptance) DFlash2 (acceptance) delta Q4_K_XL (code) 80.5 tps (94%) 95.1 tps (84%) +18% Q4_K_XL (list) 91.4 tps (98%) 127 tps (98%) +39% Q4_K_XL (prose) 53.8 tps (78%) 51.9 tps (64%) -3% Q4_K_XL (6k ctx) 80.2 tps (97%) 95.3% (90%) +19% Geometric mean 75.1 tps 87.9 tps +17% Runs were conducted after an extensive sweep of parameters for MTP and DFlash2 to find optimal settings for both drafters on Apple Silicon (M5 Max). Your settings will likely be different, but mine were n-max = 3, p-min = 0.75 for MTP and n-max = 5, p-min = 0.75 for DFlash2. At longer context (25K), DFlash2 roughly equals MTP, so there is some drop-off whereas MTP stays nearly flat. Acceptance was compared between the Q4_0 draft head and Q8_0, and there was no measurable difference in either speed or acceptance, so I'd recommend choosing Q4_0 for the memory savings. Speaking of which, in addition to being significantly faster for all categories except prose, DFlash2's draft heads are TINY! The Q4_0 head is only ~350MB, compared to 2.2GB for the Q4_0 ggml MTP head I was using. However, these savings don't translate into VRAM, because the compute buffer required for DFlash2 basically equals it back out. Finally, unlike DFlash2 for 3.8 27B, images work! The draft models drafts with vision enabled and test image prompts went through and got coherent responses with good acceptance. Overall, DFlash2 is a significant upgrade in decode performance for every type of prompt except prose, where the regression is minor and within noise.   submitted by   /u/returnity [link]   [comments]