Curious what everyone runs in practice: - What quant do you use most, and for what (chat, coding, agents)? - 2-bit: do you use it? Is it actually usable or too broken? - 3-bit: good enough for coding and more complex stuff, or do you go back to 4-bit? - If you look at KLD: what number do you call "basically lossless"? Is 0.1 already good, or do you want something like 0.01-0.03? Real experience over benchmarks. Thanks! (English isn't my first language, so I used an LLM to help translate this post.)   submitted by   /u/Queasy_Back2588 [link]   [comments]