I see everyone improving decode with bespoke models, like Sushi or Splash with Qwen 27b and Qwen Flash Next, and I'm running QFN now at 40tk/s on my M3 Max which is awesome. But prefill feels stagnant. I was trying to work on layer / attention propagation / approximating KVcache with QFN and could get like 2x prefill (400tk/s to ~850tk/s) but needle test failure started to suck. Clearly trading accuracy for speed. Is anyone working on anything clever here, or is slow prefill just a hardware limitation.   submitted by   /u/Zeeplankton [link]   [comments]