TLDR: Qwen3.8-27B UD_4_XL quant with 13.23% less KL divergence than unsloth (on my small testset, I just have a single 5090, feel free to test more). Technically it performs 4% better on my LiveCodeBench benchmark (but that would be a single data point, so it's not super-reliable). It is equally fast and using it in open-webui it "feels" better. https://huggingface.co/christophedevriese/Qwen3.8-27B-Hessian-Refined-GGUF I noticed from the MACaQ paper, which integrates changes proposed in Google/Deepmind papers, you can take the 3 components they use and just apply 1 and 2, and you just get >15% better KLD, 5% better performance on some benchmarks, while sacrificing absolutely nothing. Just switch the GGUF file. The numbers demonstrated here are the maximum that can be achieved in ~4 hours of optimizing on a single 5090, not the maximum that can be achieved. So it might be more than 15%, although I wouldn't necessarily hope for it. This can be applied, without any patches to inference software, to text models, image models, even whisper ... the works. Switch GGUF file. Done/done. The dream being that someone from unsloth looks at this and applies this to all their models (or just gives me access to a B300 machine for a day or two) The tricks used by Huawei and a short discussion of the effect: 1) Hessian-guided quantization choices: every layer is compressed differently, and the "damage" the compression does is measured. Then after seeing what damage is observed at a given level we find a combination of compressed layers that has the least damage while still achieving the same compression. For users this one is totally free. No extra compute (in fact, a tiny bit less), no extra VRAM (again, 200 bytes less), no updates to llama.cpp, works literally anywhere UD_4_XL worked, just better. 2) Recovers some lost precision by another Hessian optimization changing the weights to compensate directly for accumulated errors over multiple layers based on sample data (taken from Wikipedia). I used a Google/Deepmind version of this process that had better results. This too is totally free for users. It just changes the quant GGUF to be closer to the BF16 one. For users, just 3) random Hadamard transform + scaling: this is a trick from 3D geometry that makes all values nearly equal, and then uses an extra scaling factor to get them in optimal to better compression with quantization (yes, not technically 3D, but nD, with n=2048, but it applies in 2D and upwards). This also costs a little bit in inference performance (and requires rewriting your kernels which is quite a bit harder than I thought it was. Well, it isn't if you're ok with 20% slower inference, but you can get it down to like 0.8% with careful, days-long pondering of the problem and a lot of tries). Also very nice, but the Google version of this one is also better, but that is not the real problem. This transform also helps with compression (in fact it's kind of like what NVFP4 tries to do, just this is better) BUT requires an extra step at inference time (in fact requires 2 of them if you don't think through inference carefully, but you can "bake" one of the steps into the weights). This means that it requires patching llama.cpp and everywhere else you want the model to run (although thanks to how mlx works, you can make an mlx version that does not require updates to the inference software. But GGUF based inference software will require updates)   submitted by   /u/wapswaps [link]   [comments]