CUDA: faster MMVQ for low-bit types by am17an · Pull Request #30517 · ggml-org/llama.cpp

Read post on reddit.com

big speedup, but only for specific quantizations like Q2_K and Q3_K   submitted by   /u/jacek2023 [link]   [comments]