Sorry, yet another flipping engine! A few weeks ago, I made a speed focused experimental fork of Splash for M5 Max users recently called Splish. One of the most requested features was Qwen 3.8 Flash-Next support. Honestly, my experience with Flash-Next was underwhelming before this. 27B is such a beast that I never saw margins from Q4 versions big enough to justify the RAM footprint. Then came Sushi. What a slept on engine. Totally flipped my opinion on FN. With EXL3 trellis weights, you can fit on 5b -6/8b models and they cook. Inspired, and borrowing heavily, I wrote new kernels to see what else was left on the table. The result is Ijah. **\* With 5b-6/8b Model: Quality*:* About 4–5× closer to the bf16 model than Jundot's oQ4e (KLD 0.033/0.009/0.015 vs 0.128/0.036/0.078), ~2× closer than Sushi-4bpw. Decode*:* ~100 tok/s (90–110 through the server) — ~6% behind oMLX but with 5-bit experts. With Inco's DFlash2 drafter released today I'm getting ~120 tok/s. Will charts later.. *** So it'll lose in a drag, but honestly I think it more than makes up for it. Please give it ago and let me know how it goes? Support my token addiction ko-fi.com/severalviolins.   submitted by   /u/SeveralViolins [link]   [comments]