RustingStrata has two GPU backends that run the same kernels: CUDA (compiled at run time with NVRTC, no nvcc, no C++ build step) and Vulkan. Vulkan is how it runs on AMD, so I wanted to know what it costs on a card where I can compare both. Same file on every card: Qwen3.6-35B-A3B-UD-Q2_K_XL.gguf, fully in VRAM, MTP off. GPU Prefill, 4K prompt: CUDA / Vulkan Decode after 4K: CUDA / Vulkan Vulkan vs CUDA (prefill / decode) RTX 3060 12 GB 2216 / 2154 tok/s 113.4 / 113.6 tok/s 97% / 100% L40S 9030 / 8571 213.2 / 184.9 95% / 87% A30 3544 / 3326 131.9 / 104.0 94% / 79% A100 80 GB 6203 / 5589 173.6 / 131.9 90% / 76% H100 NVL 9131 / 6428 245.8 / 191.5 70% / 78% On the 3060 Vulkan is the same speed as CUDA. On the datacenter cards it loses 5-30% of prefill and 13-24% of decode, and its runs are noisier (A100 4K prefill ranged 4788-5990 on Vulkan against 6186-6214 on CUDA). Does it change the output? Greedy top-1 agreement with llama.cpp over 20 prompts x 32 tokens is 98-99% on both backends. How it was run: rustingstrata bench --model $M --gpu --backend cuda|vulkan --pp 512,4096 --tg 128 --reps 3 --mtp 0, mean of 3. Repo: RustingStrata   submitted by   /u/Serious_Scarcity7818 [link]   [comments]