Mellum2.1 Thinking with the MTP head Multi-token prediction head The multi-token prediction (MTP) head is one extra transformer block, trained together with the model, that predicts the token after the one just sampled. Inference engines use it as a speculative draft: the model verifies every drafted token, so the output is unchanged while decoding runs 1.4–1.8x faster, with about 90% of the drafted tokens accepted. The head lives in model-mtp.safetensors under the mtp. prefix, announced by num_nextn_predict_layers: 1 in config.json; everything else is identical to Mellum2.1 Thinking. vLLM 0.33 or newer: --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}', as in the serving commands above. transformers 5.20 or newer: pass use_mtp=True to generate. llama.cpp: use the GGUF repository with --spec-type draft-mtp --spec-draft-n-max 1.   submitted by   /u/pmttyji [link]   [comments]