Here's my *first* implementation of KVA projectors on QFN (just the uncensored model for now) the highlights are basically as follows for using the projectors at each different layer: Starting at layer 12, prompt processing speeds up 1.85x [1700 t/s -> 3150 t/s] at the tradeoff of increasing perplexity a total of +8% At layer 16, prompt processing speeds up 1.7x [1700 t/s -> 2900 t/s] at the tradeoff of increasing perplexity a total of +5% At layer 24, prompt processing speeds up 1.45x [1700 t/s -> 2500 t/s] at a tradeoff of increasing perplexity a total of +2.6% This method is different than the other KVA projectors I have seen for the following reasons: My method uses one full map per layer that uses Tikhonov regularization/ridge regression vs a per layer + training correction heads thats applied to 4 streams, then averaged out. This translates to higher accuracy and less perplexity, at the cost of more VRAM. The other methods use apx 400mb while mine uses 1.5ish GB. Other methods predict later layers keys, values, and inputs directly, while my method predicts strictly the inputs to the later layers, and depends upon the models actual weights to compute keys/values. Finally, the other methods I've seen are not variable by which layer implementation starts at (usually locked to 24 i believe), while my method is variable and allows you to determine your own risk tolerance for increasing PP speeds at the cost of increased perplexity. Here's a summary of the BetterBench results. Metric Result Detail Prefill 3,600 t/s @ 64k tok Decode 74.7 t/s Weighted combined Concurrent 70.3 t/s @ 8 streams (48/48 ok) TTFT (P50) 338 ms Single stream Update (P99) 51.5 ms Stream stutter BUT WAIT, THERE'S MORE! Here's my 2nd implementation. Based off of the HySparse2 paper, it appears that they are using a similar method but multi-layered instead of single layer. Based off of this, I've built an initial early version of this. Here's what the preliminary results show: Multi-layer - 1.55x speedup at only a +2% of perplexity Using this, I strongly believe that this can be implemented for a total of 1.5x speedup while