Seems useful for 8 & above channel workstation/server setups comes with AVX-512. I remember that few folks here do use their old servers for CPU-only inference. Also useful for Systems/Laptops with minimum 64GB RAM. Repo has more details. HuggingFace for GGUFs : https://huggingface.co/Synchopate   submitted by   /u/pmttyji [link]   [comments]