I just setup my server: 64GB DDR4 2133 4CH (Broadwell Xeon) 2 x RX 6800 16GB 1 x Rx 580 2048sp (16GB) I am using the Q4 of Qwen Flash Next, with the prompt: udo ./llama-server -m ./models/Qwnflash180b-Q4.gguf --host 0.0.0.0 --port 8081 -ngl 99 -fa on --moe-cache-mib 16384 --log-verbosity 4 --metrics -mg 1 I am able to get 19-21t/s generation at full context (262K) (which is why i capped the moe-cache-mib at 16GB) However note: when I set any context size with the -c or --ctx-size flags, performance tanked to 8t/s even when the context was smaller (100K).   submitted by   /u/IVequalsW [link]   [comments]