i Got a bit obsessed with my P40 this week trying to max speed on Ornith 1.5 35B-A3B (qwen3.5 MoE arch) without going below Q4 weights or q8 KV. Shrinking quants to win a benchmark has not really been useful to me so ai wanted a logical middle ground that was still “fast” Setup: one Tesla P40 24GB on PCIe x1 (yes, x1), Windows, my own llama.cpp fork. Q4_0 built from the official BF16, MTP draft (n=2), full 262k context, q8_0 KV. What actually moved it: - ECC off +1.5 GB VRAM and ~9% decode. - q8_0 KV, not q4_0. On Pascal q4 KV falls apart with depth (30.6 vs 42.2 t/s at 32k). - Check your MTP head. The first upload of this model, and quants built from it, have an untrained MTP head. `blk.40.nextn.shared_head_norm` mean ~1.02 = untrained, ~2.93 = trained. Draft acceptance went from ~45% to 60-85%. - int8 dp4a for the KQ half of the flash-attention tile kernel. Pascal has no tensor cores and crippled fp16, but dp4a is fast. Prefill +26% at 32k, +41% at 100k. - MTP prefill window. The MTP head is a full-attention layer, so on long prompts it costs ~13% prefill. Letting it read only the last 8k of the prompt gets that back (121k prompt: 327 -> 373 t/s). Output doesn't change, the main model still verifies every draft. Didn't work: f16 KV (faster in llama-bench, but slower and less context once MTP is on in a real server), dequantizing V inside the kernel (-5%), the official Q4_K_M (same quality score as my Q4_0, slower, less context). llama-bench, no MTP, 2 runs each: depth prefill t/s decode t/s 0 1104 60.8 8k 880 57.0 32k 552 46.0 64k 366 35.8 100k 265 29.1 190k - 18.8 Real server with MTP: 65-74 t/s on short prompts. A 101k-token prompt reads at ~415 t/s and generates at 35-39 t/s. I couldn't find anyone else posting P40 numbers at depth, so hopefully the curve helps someone. Happy to share the patches if you want to try them on your own Pascal card.   submitted by   /u/sargetun123 [link]   [comments]