Here I am with another benchmark/comparison. I was curious about the efficiency claims of Swift 1.5, UkisAI's fine-tune of Qwen3.8-Flash-Next that thinks for a shorter time. UkisAI says Swift uses 63% fewer thinking tokens with under 1% accuracy loss. I compared it with my current setup (Halogen) and Unsloth's UD-Q4_K_XL on my Asus ROG Flow Z13 (128 GB, 70 W). I ran the models on Halogen, gufo and llama.cpp, all with MTP. Since gufo can't load any of UkisAI's published 4-bit Swift GGUFs, and didnt have an XL version, I requantized their Swift Q8_0 to the same per-tensor types as Unsloth's UD-Q4_K_XL, with UkisAI's imatrix. It is even the same size as Unsloth's UD-Q4_K_XL (111.3 GB). My main use is coding agents, so I also ran 11 coding tasks through Pi and threw in some math problems and LCB tasks to compare quality and speed. Speeds at 32k filled prompt. I got very interesting results but out of 1 run so take them with a grain of salt. Here are the results sorted by total score and then total time. Engine (quant) Server start TTFT, 32k prompt Prefill t/s Decode t/s MTP acceptance KLD Math (3): score, time, tokens LCB (3): score, time, tokens Pi (11): score, time, tokens Total Halogen (v2 .hgn) 72 s 22-30 s 1,045-1,426 51-60 69-83% not published 3/3, 53 min, 154k 1/3, 61 min, 172k 11/11, 46 min, 99k 15/17, 160 min Halogen (Swift UD-Q4_K_XL requant) 33 s 27-32 s 1,002-1,186 40-43 69-86% 0.047 3/3, 75 min, 157k 1/3, 74 min, 149k 11/11, 35 min, 63k 15/17, 184 min gufo (Unsloth UD-Q4_K_XL) 25 s 20-27 s 966-1,186 42-46 64-86% 0.024* 2/3, 68 min, 154k 1/3, 75 min, 164k 11/11, 33 min, 75k 14/17, 176 min llama.cpp (Swift UD-Q4_K_XL requant) 69 s 102-106 s 300-310 20-23 41-62% 0.047 2/3, 176 min, 154k 1/3, 165 min, 139k 11/11, 74 min, 61k 14/17, 415 min gufo (Swift UD-Q4_K_XL requant) 22 s 21-28 s 911-1,186 48-49 67-81% 0.047 1/3, 55 min, 140k 1/3, 57 min, 139k 11/11, 24 min, 56k 13/17, 136 min Halogen (Swift IQ4_XS) 32 s 23-25 s 1,283-1,385 46-51 72-84% 0.129 1/3, 63 min, 146k 1/3, 71 min, 163k 11/11, 29 min, 60k 13/17, 163 min llama.cpp (Swift IQ4_XS) 188 s 99-107 s 298-320 20-23 39-61% 0.129 1/3, 149 min, 148k 1/3, 143 min, 138k 11/11, 67 min, 62k 13/17, 359 min Halogen (Unsloth UD-Q4_K_XL) 77 s 26-31 s 1,024-1,238 36-40 70-81% 0.024* 2/3, 79 min, 155k not run 11/11, 51 min, 96k 13/14, 130 min I think the gufo + Swift UD-Q4_K_XL requant might be a good fit for coding. Notes from the session (AI generated summary) For all runs retrieval was 21/21. Every setup passed all 11 Pi tasks. Swift wrote 56-63k output tokens on them against 75-99k for Qwen3.8-Flash-Next. The Swift requant finished 24-31% sooner than Qwen on the same engine, and the IQ4_XS 37-43% sooner on Halogen. Per task, the Swift requant was faster than Unsloth's UD-Q4_K_XL in 7 of 11 tasks on gufo and 8 of 11 on Halogen. gufo with the Swift requant was the fastest Pi setup: 24 min, against 33 min for Unsloth's UD-Q4_K_XL on gufo and 46 min for the Qwen .hgn on Halogen. On the math and LCB problems the token savings were small. Math used 140-157k tokens on every setup, since 20 of the 24 math runs hit the 51k thinking cap. LCB used 138-163k tokens on Swift against 164-172k on Qwen, and every setup that ran LCB hit the 60k limit on 2 of the 3 problems. Swift solved fewer math problems on gufo (1/3 against 2/3 for Unsloth's UD-Q4_K_XL) and as IQ4_XS (1/3 on both engines, against 2/3 and 3/3 for Qwen on Halogen). The Swift requant on Halogen scored 3/3, the same as the Halogen v2 .hgn. Halogen decodes its own .hgn format faster than a GGUF. Qwen3.8-Flash-Next runs at 51-60 t/s from the .hgn and 36-40 t/s from Unsloth's UD-Q4_K_XL. There is no .hgn build of Swift, so on Halogen it runs from a GGUF at 40-51 t/s. llama.cpp took 2-3x as long as Halogen and gufo on this machine: 359-415 min in total against 136-184 min. Prefill was 298-320 t/s against 911-1,426 t/s, and decode 20-23 t/s against 36-60 t/s. Model files and MTP heads v2 .hgn: Halogen's own v2 build of Qwen3.8-Flash-Next (4.16 bpw), with its own MTP draft head. Unsloth UD-Q4_K_XL: Unsloth's GGUF of Qwen3.8-Flash-Next, 111.3 GB. Halogen repacks a GGUF into its own layout at load. Swift requant: Requantized from Swift Q8_0 to the same per-tensor types as Unsloth's UD-Q4_K_XL, with UkisAI's imatrix. Swift IQ4_XS: UkisAI's file, 97.7 GB. MTP: Halogen uses its own draft head for every file. gufo and llama.cpp use Unsloth's shared Q8_0 MTP head. Upstream llama.cpp can't load that head, so the llama.cpp runs use Unsloth's fork (b11786). Where the KLD numbers come from Swift files: my numbers, against Swift's Q8_0 with llama.cpp on the wikitext-2 test set (100 x 512 tokens). The requant has 92.7% same top token, the IQ4_XS 88.1%. UkisAI's card lists the IQ4_XS at 0.1305 against Swift BF16 on the same test. * Unsloth UD-Q4_K_XL: from Rulith's EXL3 model card, against Qwen BF16 over six kinds of text at context 1024. It is not the same test as mine, so don't compare it directly with the Swift numbers. No KLD is published for the Qwen .hgn. Setup and caveats One run per problem, 17 problems in total. A difference of 1-2 problems can be chance. Speed columns: a coding-agent prompt of 32k tokens sent cold with a 64k context, then 3 follow-up turns, 3 runs. TTFT, prefill and decode are from the first turn, ranges over the 3 runs. Server start is launch to ready. Retrieval: the first turn asks for 7 const values from Rust files spread across the 32k prompt, so 21 over 3 runs. Math: AIME 2026 #15, HMMT Feb 2026 #17 and #20, with thinking capped at 51k tokens. LCB: LiveCodeBench arc196_a, arc196_c and arc196_d, with a 60k output limit and no thinking cap. Pi: 11 Aider polyglot exercises in Rust, Go and C++. Tokens are output tokens, thinking included. Sampling from the Qwen model card (temperature 1.0, top_p 0.95, top_k 20), thinking effort xhigh. I skipped LCB for Unsloth's UD-Q4_K_XL on Halogen and did not run Unsloth's UD-Q4_K_XL on llama.cpp. This whole comparison took way more time and effort than I anticipated. Versions: Halogen 0.18.13, gufo da5d5a2 (main after v0.19.0), Unsloth's llama.cpp fork b11786 for serving, upstream llama.cpp b11794 for the requant and KLD runs. Let me know if you have run Swift on other hardware and see different numbers.   submitted by   /u/deepu105 [link]   [comments]