I've been tuning llama.cpp on my Strix Halo box (Radeon 8060S, 128 GB) and the numbers are good enough to share. It's a branch called rdna35-perf on top of halo-box/strix-llama.cpp, a llama.cpp fork aimed at Strix Halo: https://github.com/asf0/strix-llama.cpp/tree/rdna35-perf (10 commits, ROCm/HIP only) TL;DR Prompt processing: 21–54% faster than Vulkan, and 8–174% faster than stock ROCm. The lead over ROCm grows with context, because stock ROCm flash attention falls off hard at depth. Generation: ties Vulkan on the dense 27B. On the MoE and the 2B it's 2–3% behind Vulkan at short context and within 1% from 64k up. 5–29% faster than stock ROCm. Quality: outputs stay closer to stock ROCm than Vulkan's do, and perplexity is unchanged. Model PP vs Vulkan (0k → 128k) PP vs stock ROCm (0k → 128k) TG vs Vulkan (0k → 128k) Qwen3.6 35B-A3B (MoE) +54% → +36% +38% → +88% −3% → 0% Qwen3.8 27B (dense) +38% → +25% +8% → +174% 0% → 0% Qwen3.5 2B (dense) +21% → +34% +9% → +87% −3% → 0% Results llama-bench, Q8_0 weights, q8_0 KV cache, flash attention on, tok/s. "Depth" is how many tokens are already in the KV cache when the test runs. Vulkan and ROCm are the same strix-llama.cpp master that rdna35-perf is built on. Qwen3.6 35B-A3B (MoE, ub 4096) Test Depth Vulkan ROCm rdna35-perf vs Vulkan vs ROCm pp4096 0 2625 2935 4045 +54% +38% pp4096 8k 2304 2430 3452 +50% +42% pp4096 16k 2065 1966 3004 +45% +53% pp4096 32k 1716 1460 2388 +39% +64% pp4096 64k 1279 959 1695 +32% +77% pp4096 128k 786 568 1068 +36% +88% tg64 0 58.2 52.3 56.3 −3% +8% tg64 8k 56.1 50.4 54.3 −3% +8% tg64 16k 54.6 48.5 53.2 −3% +10% tg64 32k 51.8 45.3 50.7 −2% +12% tg64 64k 47.1 39.7 46.4 −1% +17% tg64 128k 39.9 32.2 39.9 0% +24% Qwen3.8 27B (dense, ub 2048) Test Depth Vulkan ROCm rdna35-perf vs Vulkan vs ROCm pp2048 0 484 614 666 +38% +8% pp2048 8k 442 542 611 +38% +13% pp2048 16k 421 476 575 +37% +21% pp2048 32k 385 348 513 +33% +47% pp2048 64k 326 204 422 +29% +107% pp2048 128k 250 114 312 +25% +174% tg64 0 8.0 7.6 8.0 0% +5% tg64 8k 7.9 7.4 7.9 0% +7% tg64 16k 7.8 7.2 7.8 0% +9% tg64 32k 7.6 6.8 7.6 0% +12% tg64 64k 7.3 6.2 7.3 0% +18% tg64 128k 6.8 5.2 6.7 0% +29% Qwen3.5 2B (dense, ub 2048) Test Depth Vulkan ROCm rdna35-perf vs Vulkan vs ROCm pp2048 0 6481 7179 7852 +21% +9% pp2048 8k 5618 6253 7158 +27% +14% pp2048 16k 5000 5172 6611 +32% +28% pp2048 32k 4155 3888 5580 +34% +44% pp2048 64k 3220 2597 4277 +33% +65% pp2048 128k 2160 1547 2886 +34% +87% tg64 0 99.3 91.5 96.8 −3% +6% tg64 8k 95.9 88.2 93.2 −3% +6% tg64 16k 92.9 84.5 91.4 −2% +8% tg64 32k 88.3 79.1 87.1 −1% +10% tg64 64k 79.5 69.2 79.2 0% +14% tg64 128k 67.6 56.2 67.7 0% +21% f16 KV gives the same picture: PP 20–56% faster than Vulkan and up to +171% over stock ROCm at 128k, TG within −3% / +2% of Vulkan. Quality rdna35-perf changes kernels, so I checked it doesn't change outputs: KL divergence against stock ROCm on wikitext-2 (6 × 4096-token chunks), plus perplexity at 32k context. Model KLD vs stock ROCm (Vulkan / rdna35-perf) Same top token (Vulkan / rdna35-perf) PPL @ 32k (Vulkan / ROCm / rdna35-perf) Qwen3.6 35B-A3B 0.0038 / 0.0018 97.8% / 98.5% 6.028 / 6.023 / 6.024 Qwen3.8 27B 0.0066 / 0.0013 98.9% / 99.7% 6.003 / 6.004 / 6.007 Qwen3.5 2B 0.0008 / 0.0001 98.6% / 99.5% 10.046 / 10.049 / 10.049 rdna35-perf is closer to stock ROCm than Vulkan is, and perplexity is the same across all three builds. What changed Flash attention kernel for RDNA3.5 (fattn-rdna35). This is the biggest win and the reason long-context PP holds up. GGML_FA_RDNA35=0 turns it off if you want to compare. Prefill GEMM tuning for the quantized matmuls. MoE: routing helper, fused reduction + residual, grouped mat-vec for speculative-decoding verify batches. Gated DeltaNet (the linear-attention layers in Qwen3.5/3.6/3.8): fused SiLU-gated output norm and fused decode. Decode: plain mat-vec for rows longer than 4096 (this closed the 27B decode gap to Vulkan), plus decode fusions for the Qwen3.6 MoE. Misc: HIP graphs off by default on RDNA3.5 (slower here), a faster penalties sampler, host-side prefill for Qwen3.5/3.6 with a draft model, and DFlash drafts read target features in place on the GPU. I have only tried those 3 models so far.   submitted by   /u/asfbrz96 [link]   [comments]