I built a browser LLM engine that's specialized per model rather than generic: Rust compiled to wasm, plus hand-written WGSL kernels for Qwen3.5-2B (Gated DeltaNet + gated attention) and MiniCPM5-2B. Same model, same chat prompts, greedy, Chrome 156: webyIQ WebLLM 0.2.87 transformers.js 4.3 RTX 5080 generation 620 tok/s (417 without speculation) 94 31 RTX 5080, ~1k-token prompt 0.20 s 0.81 s ~129 s AMD 2-CU iGPU generation 37 tok/s 4.7 crashed What's in it: fused int4 (GPTQ, group 32) × int8 GEMV/GEMM, with dot4I8Packed and subgroups; a decode loop that stays on the GPU (argmax on GPU, pipelined readback); speculative decoding with Qwen3.5's built-in MTP head. Drafts are verified on the GPU, so the output is identical to greedy. It turns itself off where it doesn't pay, e.g. on the iGPU; weights at +2.4% WikiText perplexity vs bf16, 1.1 GB. Fun side effect: at this speed a 2B model can drive web pages through the DOM at 0.2–0.4 s per step. There's a browser-agent demo and a side-panel extension for Chrome/Edge. Caveats: it's a prototype, tested on Windows, and Safari/mobile are untested. Demo: https://rtx527896828.github.io/webyIQ/ · Code (Apache-2.0): https://github.com/rtx527896828/webyIQ · Weights: https://huggingface.co/tixiren/Qwen3.5-2B-webyIQ In depth: https://medium.com/@tixiren/running-a-2b-llm-at-600-tokens-s-in-the-browser-e5237f99c8ff   submitted by   /u/tixiren [link]   [comments]