Hi everyone! I've been experimenting with Strata for a while now, running it on a system with a Ryzen 7 9800X3D, RTX 4090, and 64 GB of RAM. I'm currently using Qwen Flash Next, and I have a few questions about model quantization. Since my hardware is right at the edge of what's practical for running this model, I'd love to hear about your experiences with different quantization formats. I've come across discussions suggesting that GSQ-RCO IQ3_S might deliver better response quality than Unsloth UD IQ4_XS. Is that actually the case? What have your experiences been with these two quantizations? Are there any other quantizations that might offer better quality while still being feasible to run on my hardware? In theory, Unsloth UD-Q4_K_XL also works (according to Strata, around 6 GB of the model is loaded from NVMe), but the generation speed drops to around 20-30 tokens/s, which makes it somewhat impractical for extended use. I'm also curious how NVFP4 quantizations compare to the ones mentioned above in terms of response quality and performance. Thanks in advance for sharing your experiences and recommendations!   submitted by   /u/Jagerius [link]   [comments]