Looks quite interesting. Lots of numbers presented, but for a single model only. Highlights: File sizes are the same as Q-quants. KLD improves substantially over Q-quants. Performance suffers slightly Creating IQ quants are somewhat slow. Asterisks apply... https://github.com/ggml-org/llama.cpp/pull/30715 https://huggingface.co/eaddario/IQ2_K-IQ6_K-Test-Models Appologies to Mr. Addario if I mangled anything.   submitted by   /u/ethertype [link]   [comments]