Hey everyone, China Telecom (TeleAI) recently released Xing4.0-29B-A4B (a 29.5B MoE with ~4B active parameters, combining DeepSeek-style MLA and 4-stream mHC residual mixing): Model: https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B Upstream llama.cpp PRs are currently stalled or closed due to debates around adding model-specific GGML opcodes. To get the model running cleanly without adding technical debt to the engine, I've opened PR #2808 on ik_llama.cpp providing full native runtime support and an HF-to-GGUF converter: PR: https://github.com/ikawrakow/ik_llama.cpp/pull/2808 What's implemented in this PR: Full Architecture Runtime (LLM_ARCH_XING4_0): 4-stream mHC: built entirely using standard generic GGML primitives (pre/post blocks, affine transforms, clamp [-30, 30], Sinkhorn iterations). Zero new ggml_op added, so it runs out of the box on CPU, CUDA, Metal, and Vulkan. Stream pooling: unweighted average across the 4 streams before output norm. MLA attention: DeepSeek-V2 style latent attention with dynamic head sizing. MoE FFN: dense lead layers + routed experts + shared expert. Node budget: bumped max_nodes (n_tokens * 160) to prevent graph overflow with large ubatches (ubatch >= 2048). HF to GGUF Converter: Adds Xing4Model to convert_hf_to_gguf.py with native SentencePiece extraction from tokenizer.model. Direct tensor mapping for mHC weights. Quantization: Automatically protects hc_attn_fn and hc_ffn_fn from quantization in llama-quantize, and forces 1D scales/bases to F32. Quantization floor: keeps MLA latent projections at source precision for Q6_K and Q8_0 to avoid quality loss. Verification: Perplexity: 12.4190, generation is coherent with no NaNs. Full chat template included with thinking tags (...) and tool calling. Quick test from the branch: git clone https://github.com/ikawrakow/ik_llama.cpp cd ik_llama.cpp git fetch origin pull/2808/head:xing4 && git checkout xing4 cmake -B build -DGGML_NATIVE=ON cmake --build build --config Release -j$(nproc) python3 convert_hf_to_gguf.py path/to/Xing4.0-29B-A4B --outtype f16 --outfile xing4-f16.gguf ./build/bin/llama-cli -m xing4-f16.gguf -p "Hello!" -n 64 Feel free to test it out and report back if you get benchmarks on different setups!   submitted by   /u/Any_Bus9205 [link]   [comments]