Hello everyone! As always, thank you for sharing you previous results (and bugs) on the last version. I am happy to report that LlamAmpere v0.6 is now out. The headline items for you (at temperature 1): a ~13% jump in TPS, with a 112 TPS average across 3 task types in the 100-105K token interval a +20% increase on prompt processing, now starting around 1800 TPS and holding 1000+ through the first 100k The average prefill over the first 100K went from ~1,080 to ~1,300 TPS A new file format (Ampere Integer+) designed to be as fast, smaller, and higher precision (lower KLD) than XS formats. Vs EXL3, it is approximately the same KLD, with a significantly faster speed. The recommended model (now available on HF) is about 10% better KLD and 1.2 GB smaller. This results in slighter faster speed, and of course more room for context. I will be releasing more quants at smaller sizes in this format later this week as I continue to tweak the design at 2 and 3 bits. I have also upgraded EXL3's kernels, and in my testing it was running ~8% faster than in the original EXL fork. But, at 4 bits this configuration shows ~17% faster decode and 25% faster prefill on AI+, so I will be focused more on those kernels moving forward (EXL had a non-stat sign