I got qfn working on my 5090 + 64gb ddr using basalt with 256k int8 kv and had been testing it out. Based on vibes it seemed worse than 27b but I kept seeing posts of people saying qfn is so much better so I thought I'd test the 'best possible' setups of each model for my 5090 32gb + 64gb ddr system. Qwen Flash Next using basalt, nvfp4 quantization, 256k int8 kv cache, one slot. Qwen 3.8 27b nvfp4 quasar qat, 570k nvfp4 kv cache pool using ninfer, four slots. Summary Results Qwen Flash Next BFCL 78.19% LiveCodeBench 79.22% Qwen 27b BFCL 81.51% LiveCodeBench 86.15% 27b dominates in the long context, harder to solve problems. QFN had a small advantage in the short single answer categories. To me this is a conclusive result that also aligns with my own experience. I'm glad it turned out this way, having 4 concurrent agents and a huge kv cache pool is just too good to give up. Detailed results below: Qwen Flash Next: BFCL category accuracy correct/total 95% CI simple_python 92.50% 370/400 89.5–94.7% multiple 92.00% 184/200 87.4–95.0% parallel 86.50% 173/200 81.1–90.6% parallel_multiple 78.50% 157/200 72.3–83.6% live_simple 75.97% 196/258 70.4–80.8% irrelevance 76.25% 183/240 70.5–81.2% multi_turn_base 56.00% 112/200 49.1–62.7% multi_turn_long_context 54.50% 109/200 47.6–61.3% BFCL all (item-weighted) 78.19% 1484/1898 76.3–80.0% LiveCodeBench value problems (graded) 231 / 231 pass@1 79.22% empty answers (counted as fails) 27 pass@1 easy 100.00% (57) pass@1 hard 59.00% (100) pass@1 medium 90.54% (74) pass@1 atcoder 67.86% (140) pass@1 leetcode 96.70% (91) Qwen 3.8 27b BFCL category accuracy correct/total 95% CI simple_python 89.75% 359/400 86.4–92.4% multiple 91.00% 182/200 86.2–94.2% parallel 91.50% 183/200 86.8–94.6% parallel_multiple 83.50% 167/200 77.7–88.0% live_simple 83.72% 216/258 78.7–87.7% irrelevance 79.58% 191/240 74.0–84.2% multi_turn_base 64.00% 128/200 57.1–70.3% multi_turn_long_context 60.50% 121/200 53.6–67.0% BFCL all (item-weighted) 81.51% 1547/1898 79.7–83.2% LiveCodeBench value problems (graded) 231 / 231 pass@1 86.15% pass@1 easy 98.25% (57) pass@1 hard 75.00% (100) pass@1 medium 91.89% (74) pass@1 atcoder 80.00% (140) pass@1 leetcode 95.60% (91)   submitted by   /u/Fragrant_Scale6456 [link]   [comments]