Hello! We've been busy at Telperion with a lot of experimental research (more coming soon). Today, we're announcing the release of our latent attention variants of Qwen3.8-27B. Our latent attention variants (quants calibrated to FP8 KV cache) compress the KV cache by 4×, enabling a near 4× increase in maximum context on consumer hardware or a near 4× increase in concurrency on server hardware. These variants demonstrate great performance, with a statistically distinguishable increase (6.58%) in SWE performance on our MLA variant, in addition to using upwards of 20% fewer reasoning tokens. Due to the memory downsides of serving MLA on multi-GPU devices, we produced a grouped latent variant using two groups, enabling 4× KV compression for dual GPU users. |Model|SWE-bench Verified (n > 1500)|Mean Output Tokens|KV compression (TP = 1 | TP = 2)| |:-|:-|:-|:-| |Qwen3.8-27B|39.5%|—|—| |Qwen3.8-27B-MLA|42.1%|0.88×|4.00× | 2.28×| |Qwen3.8-27B-GLA-g2|39.4%|0.86×|4.00× | 4.00×| We have fantastic recipes for post-training quantization and have produced quantized versions that exceed equivalently sized .gguf quants in fidelity. Our models have an easy to install vLLM plugin. Quant Memory (including MTP head and vision encoder) Increase in top-1 disagreement (lower is better) BF16 55.7GB — FP8 31.5GB 0.06% INT4 22.9GB 0.17% EXL3-4.0bwp* 17.2GB 0.46% *EXL3 requires a custom vLLM plugin to run. The plugin has been verified on RTX PRO 6000s, H100s, and 2× RTX 5060Ti 16GB. We do not believe in gatekeeping ideas or research. As a result we will be open sourcing our method soon. This method requires zero-retraining, and is compatible with almost any model that uses grouped query attention or similar variants. Finally, we plan to make an improved version 2, with a full benchmark suite when we get more compute. We will also be making sparse latent attention variants of Qwen4-27B when released. What models would you like to see with latent attention? For those interested in following our research, learning/doing research, or getting on contact, feel free to join our discord server.   submitted by   /u/fulgencio_batista [link]   [comments]