Tested on an RTX 3060 with native Dawn WebGPU. llama-bench -m MODEL -p 512 -n 128 -r 5 -ngl 99 -dev WebGPU -t 2 -fa off Model Test master (t/s) PR (t/s) Ternary Bonsai 1.7B Q2_0 pp512 35.03 ± 0.46 1915.91 ± 3.36 Ternary Bonsai 1.7B Q2_0 tg128 14.02 ± 0.16 94.75 ± 0.20 Bonsai 2 2B Q2_0 pp512 36.40 ± 0.46 2039.89 ± 10.03 Bonsai 2 2B Q2_0 tg128 12.57 ± 0.20 71.72 ± 0.24   submitted by   /u/pmttyji [link]   [comments]