Optimization 1: Humming MXFP4 MoE backend Optimization 2: Online FP8 LM head quantization Optimization 3: MTP speculative decoding with NEXTN Configuration TPOT Decode throughput Relative to baseline Baseline 27.71 ms 36.09 tok/s — + Humming MoE backend 25.35 ms 39.45 tok/s +9.3% + Online FP8 LM head 22.76 ms 43.94 tok/s +21.7% + MTP (NEXTN, steps=3 / topk=1 / draft_tokens=4) 9.72 ms 102.88 tok/s +185.1%   submitted by   /u/pmttyji [link]   [comments]