I made a hipfire-native MQ4 of KRAFTON's Raon-1-mini-preview and measured it on a 20 GB 7900 XT (ROCm 7.1). If you just want to use this model, use KRAFTON's llama.cpp branch with the official GGUF. This build is much slower (bummer!) Maybe the hipfire team can optimize for it in the future. ``` pp512 pp8192 decode llama.cpp (Q4_K_M) 2,055 t/s 1,477 t/s 47.4 t/s hipfire MQ4 12.6 t/s 6.3 t/s 4.6 t/s ``` hipfire numbers are single runs. The likely cause is that hipfire's Raon path handles recurrent state token by token, with no batched prefill kernel. Other architectures in hipfire run at normal speed. What it has going for it: - The file is 1.29 GB smaller than the official GGUF. - The whole hybrid (RDA linear attention + gated MLA + mHC streams + 128-expert MoE) runs natively, and I checked each module against the reference fp32 code. - Agreement with the GGUF is mean KL 2.06 and top-1 51.6% (teacher-forced). I haven't run any benchmark suite, so this says nothing about absolute quality. Problems: - Stock hipfire v0.4.2.1 serve refuses every generation ("stream must begin with gen_start") — you need my patch, in the repo. - Thinking often never closes (answers land in reasoning_content), and turning it off renders a broken frame for this arch. - It failed my needle probes at 2K and 8K on hipfire with prompt-echo repetition — but the same probe on the official GGUF at 26,962 tokens recalled the passcode fine, no echo in that run. So it looks like an engine/checkpoint interaction at greedy temperature rather than a context failure, and the sampling-path difference between the two engines is worth a look. - 65K context is the ceiling on 20 GB (98K fails loading the recurrent state). I'm posting this as a before-state for the Raon kernels. If anyone wants to work on batched prefill, the numbers are reproducible — the repo has the exact patch plus the untracked kernel files, verified to apply and compile clean against the same base commit. Card: https://huggingface.co/datayoda/Raon-1-mini-preview-hipfire-MQ4   submitted by   /u/Queasy_Asparagus69 [link]   [comments]