Here it is (I think you change llama serve to llama-server for Linux). Enjoy! llama serve -hf peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF-MTP:UD-IQ3_XXS \ -ngl 99 --n-cpu-moe 30 -c 131072 --no-mmproj --parallel 1 \ -ctk q4_0 -ctv q4_0 -fa on -b 512 -ub 128 \ --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.1 \ --temp 0.6 --top-p 0.95 --top-k 20 # change these to whatever you please This is using the latest llama.cpp build. It gets about 35 tok/s prefill and 23 tok/s decode at 4K consistently on my machine. I should mention this does depend on how much RAM you are using/have free so that your OS can like do its magic BTS mmap cache stuff. Also change --parallel 1 to whatever concurrency you want. I don't really fuck with multiple agents or agent swarms (yet) and I have no clue if people actually use this (and would use it for a local model that barely runs) so I've set it to 1. I had OpenCode with the new stealth model sweep overnight and it found this. I don't really feel like making a proper write-up, but I am going to run the performance benchmark on my set of 13 prompts (listed below) and will post a table containing everything in the comments in about an hour's time as of writing.   submitted by   /u/Aggravating-Push-207 [link]   [comments]